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SYSTEM FOR LOCATING RELATIVE POSITIONS OF OBJECTS 



This application is a continuation in part of 
applications serial numbers 08/052,042 and 08/052,045, 
5 both filed April 22, 19S3. 

FIELD OF THE INVENTION 

This invention relates to locating the position 
of one object relative to another object in a three- 
dimensional space. It inore particularly refers to a 
10 systeE for locating and displaying the position and the 

orientation of a first noving object relative to a second 
moving object in three dimensional space. 

Description of Prior Art 

Computed tocography (CT) , magnetic resonance 

15 imaging (MRI) , and other methods provide important 

detailed images of the internals of human medical patients 
which are useful for diagnostic purposes. Often, these 
diagnostic tools, which are used to determine and display 
images of the inside of patients' bodies, were used, and 

20 the images they create were taken at times other than 
during actual surgical work on the patients, that is 
before and/or sometimes after surgery. Specifically, it 
is usual for CT or MRI scans to be taken before the 
surgeon starts his work. They are diagnostic tools, not 

25 tools for following an operation in progress. These prior 
scans are then commonly used to plan the surgery or at 
least to assist the surgeon in deciding what surgical 
course of action should be initiated and then followed. 
Sometimes, they are also used after surgery to determine 

30 and evaluate the results of the surgical procedure. 

If surgery is in areas of the body which are not 
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readily visible to the surgeon, such as inside the cranium 
within the recesses of the brain, it is difficult, if not 
impossible to follow the course of the surgery while it is 
"going on. It is presently impossible to conduct surgery 

5 while sinultaneously taking an MRI. Therefore, the 
surgeon is always working with previously taken images of 
the internal structures of the patient. 

While working with these previously taken images 
during surgery, there often is no obvious, clear-cut 

10 relationship between points of interest in the diagnostic 
inages and the corresponding points on the actual patient. 
While anomalous tissue may be obviously distinct from 
normal healthy tissue in the images, the difference nay 
not be so visible in the patient on the operating table, 

15 Furthermore, in intracranial surgery, the region of 
interest may not always be accessible to direct view. 
Thus there exists a need for apparatus and systems to help 
a surgeon relate locations in the diagnostic images to the 
corresponding locations in the actual anatomy of the 

20 patient, and vice versa - 

The related prior art includes references which 
disclose subject matter which describe means to accomplish 
objects which are similar to those of the present 
invention as a whole. These references include 

25 publications describing the correlation of previously 
taken internal medical images of a patient, which are 
usually three-dimensional, with the corresponding actual, 
present time physical locations on and in the patient in 
the operating room during surgery. U.S. Patent 4,791,934 

30 describes a semi-automated system which makes this 
correlation, but note should be taken that the system 
described in this patent also requires additional 
radiographic imaging to be accomplished in the operating 
room at the time of surgery, and it then also requires 

35 that these present time images be correlated into the 
coordinate systems of the previously taken diagnostic 
images so that they can be related to the live patient. 

Furthermore, the system of this 'S34 patent uses 
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a con;puter-driven robot ar:n to position a surgical tool in 
relation to these inages. It does not measure and define 
the present time location and orientation of an input 
TDrobe (a surgical tool) positioned interactively by the 
5 surgeon, and superinipose such present time data on a 
previously taken inage. 

There have been other attempts to solve the 
three-dinensionai localization problem specifically for 
stereotactic surgery. One class of solutions has been the 

10 use of a variety of mechanical frames, holders, or 
protractors for surgery (usually intracranial surgery) . 
For examples see U.S. Patents 4,931,056, 4,875,478, 
4,841,967, 4,809,694, 4,805,615, 4,723,544, 4,706,665, and 
4,651,732, 4,638,798. Generally these patents disclose 

15 systems which are intended to reproduce angles derived 

from the analysis of internal images, and most require 
rigidly screwing a frame to the skull. In any case, these 
methods are all inconvenient, time-consuming, and prone to 
human error. 

20 A more interactive method is disclosed in United 

States patent 4,750,487. This patent discloses the use of 
present time fluoroscopy in the operating room to help 
guide surgical tools. It will be abundantly clear that 
subjecting a patient to fluoroscopy is undesirable at any 

25 time. It may impose even greater risks during surgery. 

More relevant prior art are the publications 

which describe systems which have been built specifically 

for stereotactic surgery. The following reference is 

illustrative and pertinent: 

30 David W. Roberts, M.D., et al ; "A 

Frameless Stereotaxic Integration of 
Computerized Tomographic Imaging and the 
Operating Microscope", J. Neurosurgery 65 . Oct. 
1986. 

35 This reference reports on the use of a three-dimensional 
digitizer to track the position and orientation of the 
field of view of a surgical microscope and means to 
superimpose, on the field of view in the microscope, the 
corresponding internal planar slice of a previously 
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obtained computed tomographic (CT) image. Such a systen-j 
was a great step forward in this art. It was, however, 
not without attendant disadvantages. 

In the first place, the system described by 
5 Roberts et al. used the transmission of sound impulses to 
coniaunicate the location and orientation of the field of 
the surgical microscope. The use of sound is necessarily 
based on the determination of transmission distances as a 
function of the speed of sound, and the differentiation of 

10 snail distance differences. It is well known that the 
speed of sound varies to a substantial extent as a 
function of the temperature in the medium through which 
the sound travels. With modern air conditioning and 
heating, there are many different thenaoclines and air 

15 currents present in an operating room, which are of little 

concern to a patient or to the doctors, but which can 
materially effect the accurate measurement of precise 
distances as a function of the speed of sound 
transmission. Therefore very accurate compensation 

2 0 factors must be applied in order to get accurate 

information on the exact location and orientation of the 
operating microscope. It will be appreciated that small 
errors in this field can have very serious consequences to 
the patient. 

25 It must be remembered that the purpose of 

measuring distances from the sound emitters to the 
receivers is to determine precisely where the point of a 
surgical probe, or other device, is. The point may move 
only a very short distance, millimeters or even less, and 

30 so the position of the sound emitter on the probe my 
change by only a very small amount. This mensuration 
system must be able to detect such small changes in 
position and orientation of the sound emitters, and to 
translate these small changes into meaningful changes in 

35 the defined position and orientation of the point of the 
probe. • Thus' the accuracy of measuring and knowing the 
speed of sound in the medium of the operating room is 
critical to locating the point of the probe with any 
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accuracy and cress correlating this location with a 
previously taken image which corresponds to the location 
of the probe tip. It should therefore be clear that the 
-raaj or' disadvantage for this reported ''sonic system is the 
5 inherent inaccuracy and instability of the sonic 
mensuration apparatus - 

In addition to the difficulty of making accurate 
and precise distance measurements using sound 
transmission, there was another difficulty and uncertainty 

10 which was inherent in the Roberts et al. system regardless 
of its use of sound sensing means or not. The Roberts et 
al. system relied on the position of the patient, at least 
so much of the patient as corresponded to the area being 
operated on-that is for example the head, being absolutely 

15 fixed. That system was not capable of determining changes 

in the location of the patient housing the operating 
microscope, but tracked only the location of the 
microscope independent of the patient. The location and 
orientation of the patient (the head of the patient) was 

20 determined at the start of the operation and the 

previously taken CT scan related to that specific position 
and orientation. If this position and/or orientation of 
the patient was changed, intentionally or even 
inadvertently, during the entire operation, the system had 

25 to be re-correlated. That is, the operation had to be 

stopped and the previously taken CT scan had to be 
correlated with the new position and/or orientation of the 
patient's head- Clearly this was a major disadvantage of 
the Roberts et al, system. Not only does it require that 

30 the system be re-correlated at the time that the surgeon 
intentionally moved the patient's head, but it was not 
capable of taking into consideration inadvertent movements 
of the patient's head during the operation. 

The present invention does not include the 

35 taking of suitable images of the internals of a patient 
before the operation. It starts from the point at which 
these images have already been taken and are found to be 
acceptable by the surgeon. This invention therefore does 
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not coniprise the iniaging apparatus used to generate the 
internal three-dimensional izuace or F^odel of the internals 
of the patient or other object. However, this invention 
"does use these previous imaging data and inputs this 
5 information into the instant system. The system of this 
invention does not include the means of taking these 
images, but it does include the images themselves, 
preferably in electronic form. 

It is contemplated that such known imaging 

10 devices night be ultra-sound, computed tomography (CT) or 
magnetic resonance imaging (KRI) . It is also contemplated 
that such imaging device might be one which has as yet not 
been developed. The important limiting factor in the 
sense of the imager is that the data generated by that 

15 imager must be available in an electronic digital fonriat, 

or is readily convertible into such format. The digital 
data may be derived directly from such an inager and 
transmitted to the instant system over a conventional 
communication network or through magnetic tape or disk 

20 media. 

There is another area of prior art references 
which may be applicable to the patentability of the 
instant invention. This prior art is related specifically 
to localizing devices, which measure the relative 

25 positions of a manually maneuvered probe and another 
object, but not necessarily applied to "seeing" the 
present time position of the probe internal to a patient. 
Previous methods and devices have been utilized to sense 
the position of an object in three-dimensional space. 

30 These known techniques employ various methods of 
mensuration. 

Numerous three-dimensional mensuration methods 
project a thin beam or a plane of light onto an object and 
optically sense where the light intersects the object. 

35 Examples of several United States patents which disclose 
such simple 'distance range-finding devices ' using this 
general approach are: U.S. patents 4,660,970, 4,701,049, 
4^705,395, 4,709,156, 4,733,9 69, 4,743,770, 4,753,528, 
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4,761,072, 4,764,016, 4,782,239, and 4,825,091. Examples 
of United States patents which disclose using the plane of 
light to sense an object's shape include: U.S. patents 
-4,821,200, 4,701,047, 4,705,401, 4, '737, 03 2, 4,745,290, 
5 4,794,262, 4,821,200, 4,743,771, and 4,822,163. In the 

latter, the accuracy of detemining the location and 
orientation of the surface sample points is limited by the 
typically low resolution of two-dimensional sensors which 
have usually been employed (currently the accuracy of 

10 these devices is about 1 part in 512 for a solid state 
video camera) . Furthermore, these devices are noz 
presently capable of detecting the location and 
orientation of the tip of a probe whereby the tip of the 
probe identified is coincident with specific points, and 

15 identifying the location of such probe tip is synonymous 

with identifying such specific point. Additionally, 
optical systems are traditionally limited by line-of-sight 
considerations. If you cannot see rhe point in question, 
light cannot be directly impinged on that point or 

20 projected from it. Because of these inherent limitations, 

these known devices have been generally useless for 
locating a point within a recess, which is necessary for 
intracranial surgery. 

The internal imaging devices themselves (such as 

25 computed tomography, magnetic resonance imaging, or 

ultrasonic imaging) are unsuited for tracking the spatial 
location and orientation of a manually held probe during 
an operation, even though they are unencumbered by line- 
of -sight restrictions. Thus, these systems are not 

30 capable of being used to previously record an image, or a 
set of images, of the internals of a patient, and also to 
image these same internals in present time during an 
operation. 

Other prior art methods and apparatus are Jcnown 
35 which track the position of one or more specific moveable 
points in three-dimensional space. In these known 
techniques, the rtioveable points are generally represented 
by small radiating emitters which move relative to fixed 
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position sensors. Sor.e methods interchange the roles of 
the emitters and sensors. The typical forms of radiation 
are light (U.S. Fatenr 4,836,778 for exanple) , sound (U.S. 
'Patent 3, 821, < 69), and magnetic fields (U.S. Patent 
5 3,983,474). Other laethods include mechanical arms or 

cables (U.S. Patent 4,779,212). Some electro-optical 
approaches use a pair of video cameras plus a computer to 
calculate the position of homologous points in a pair of 
stereographic video images (for example, U.S. Patents 
10 4,836,778 or 4,829,373). The points of interest may be 
passive reflectors or flashing light emitters. The use of 
light emitters tend to simplify finding, distinguishing, 
and calculating the location and orientation of the 
points. 

15 Probes with a pointing tip and sonic localizing 

emitters on them have been publicly marketed for several 
years. The instant invention is also concerned with 
determining the location and orientation of a stylus, but 
it is an improvement over the known devices in that it 

20 employs tiny light emitters, in place of the known sound 

emitters. Further, as will become apparent, the method 
used to sense the positions of these light emitters is 
different from what has been used in connection with 
sound. 

25 Additional prior art related to the instant 

invention is found in these references: 

Fuchs, H.; Duran, J.; Johnson, B. ; "Acquisition 
and Modeling of Human Body Form Data", Proc. SPIE, v 166, 
(1978), p 94-102. 

30 Hesqui, F. ; Xaeser, P.; Fischer, P.; "Real- 

time, Non-invasive Recording and 3-d Display of the 
Functional Movements of an Arbitrary Mandible Point", 
SPIE Biostereometrics 602, 1985, p 77-84. 

Yamashita, Y.; Suzuki, N. ; Oshima, M.; 
35 "Three-Dimensional Stereometric Measurement 

System Using Optical Scanners, cylindrical 
Lenses, and Line Sensors, Proc. SPIE, v. 
361, 198-3, p, 67-73. 
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The paper by Fuchs, et al , (1978) best describes 
a Eiethod used to track a surgical probe in three- 
dinensional space. This nethod could possibly be used in 
"conjunction with and to supplement the practice of the 
5 instant invention. It is based on using three or more 
one-dimensional sensors, each consisting of a cylindrical 
lens and a linear array of radiation detectors, such as a 
charge-coupled semiconductor device (CCD) or a 
differential-voltage position sensitive detecror (PSD) . 

10 The sensors determine intersecting planes which all 
correspond to a single point radiating light emitter. 
Calculation of the point of intersection of the planes 
gives the location of the emitter. The calculation is 
based on the locations, orientations, and other details 

15 concerning the one-dimensional sensors and is a 

straightforward application of analytic gecmetry. This 
photo-optical method, however, has not been previously 
used for the purpose of the present invention. In that 
sense, iz is possible that the instant system could be 

20 considered to be a new and unobvious use of an existing 

system. 

Thus, there still remains a need for a complete 
system (apparatus and method) which provides fast, 
accurate, safe, and convenient mensuration of the three- 

25 dimensional position and orientation of a manually 
operated probe relative to a moveable object of interest. 
This system must also visually relate, in the same 
coordinate system, the relative position and orientation 
of the probe, even a portion of the probe, which is out of 

30 line of sight, to an image of a previously-generated 
three-dimensional model of the object, 

objects and suzuiiary of the Invention 

One objective of the present invention is to 
provide means for accurate three-dimensional mensuration 
3 5 of the relative position and orientation of a moveable 
member with respect to a moveable object. 
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Another object of this invention is to provide 
accurate visual relationship between two objects which are 
each r.oveable with respect to each other as well as with 
* respect to the coordinate system in 'which these movable 
5 objects reside. 

A further object of this invention is to provide 
accurate spacial relationships between a moving probe and 
a moving surgical patient during an operation in an 
operating room, wherein the probe and the patient are 
10 moving relative to each other as well as relative to a 
fixed location and orientation of the mensuration 
apparatus. 

A still further object of this invention is to 
provide an electro-optical mensuration system which is 
15 inexpensive, easy to use, reliable, and portable, and 

which employs a manually positioned probe, or other 
instrument, at least part of which is not within a line of 
sight of the surgeon, and which further employs a means of 
measuring the otherwise "invisible" position and 
20 orientation of the probe tip. 

Another object of this invention is to provide 
a simple, non-invasive system for establishing a 
correspondence between a presently existing coordinate 
system containing a movable object and a previously 
25 obtained coordinate system containing a three-dimensional 
computer model of that object, where the previously 
obtained computer model is also of this same system. 

Another object of this invention is to relate a 
measured location on the outside, or inside, of an object 
30 to its corresponding location in a previously generated 
computer model of that object by establishing 
correspondence between the coordinate systems of the 
object and the model. 

Another object of this invention is to display 
35 a cut-away view or a cross-sectional slice of a previously 
generated computer model of a planar cross-section of a 
geometric model, where the slice approximately intersects 
the location in the model corresponding to a location 
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measured in present time, and to superimpose a marker on 
the displayed slice to indicate the location on the slice 
corresponding to the measured location. 

Another object of this invention is to assist an 
5 operating surgeon locate subcutaneous diseased tissue 

while avoiding healthy critical structures, especially in 
cranial neurosurgery. 

Additional objects, advantages, and novel 
features of the invention shall be set forth, at least in 

10 part in the following description. These will also become 

apparent to those skilled in the art upon examination of 
the following discussion or nay be learned by the practice 
of the invention. The objects and the advantages of the 
invention may be realized and attained by means of the 

15 instrumentalities and in the combinations particularly, 

pointed out in the appended claims. 

To achieve the foregoing and other objects, and 
in accordance with the invention, as embodied and broadly 
described herein, the relative position and orientation of 

20 two or more objects, moving independently of each other in 
space, is determined using the apparatus and the method of 
this invention. As used herein, a two moving-object set, 
which is a simple illustration of the system used in this 
• invention, is defined as: 

25 at least one first movable object, which may be a 

hand held probe having an invasive tip, for touching or 
for inserting into a second object; 

at least one second movable object with respect to 
which the first object is movable, or is moving? 

30 a present time coordinate system, that is a 

coordinate system which exists at the time of determining 
the spatial interrelationship of the several moving 
objects, which includes the second object, and in which 
said second object is moving, or can be moved; 

35 a previously taken predetermined three-dimensional 

geometrical model, suitably a computer generated model, of 
the second object suitably provided in an electronically- 
accessible data base form; 
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a previous tiiTie coordinate system, which includes the 
previously taken coniputer model, that is a coordinate 
systea which was determined at the time that the data fron 
-which the computer generated model were taken; 
5 means to relate the previous time coordinate system 

of the previously taken model of the second object to the 
present time coordinate system of the second object 
itself; 

at least two first radiation emitters, mounted in 
10 spaced relation to each other on a portion of the first 
object which, during movement of said first object 
relative to said second object, are not obscured from 
receiver (s) adapted to receive the radiation emitted 
thereby; 

15 at least three second, additional radiation emitters, 

dif f erentiatable from each orher and from the first 
radiation emitters, which are mounted in fixed 
relationship to, and are movable with, the second object 
and which are not obscured from receiver (s) adapted to 

20 receive the radiation emitted therefrom 

at least three radiation sensors, the positions of 
which are known with respect to a present time fixed 
coordinate system, which are so positioned that they can 
detect the positions of at least two of the first 

25 radiation emitters and at least three of the second 
radiation emitters; 

means to differentiate between the first and second 
radiation emitters sufficient to be able to distinguish 
between the several emitters; 
' 30 means to control the number of contacts between said 

first radiation emitters and said sensors per unit of time 
sufficient to track movement of said first object relative 
to said present time coordinate system in which said 
second object resides; 

35 means to control the number of contacts between said 

second radiation emitters and said sensors per unit of 
tine sufficient to track movement of said second object 
relative to the present time coordinate system in which it 
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resides without interference with the radiation emitted 
from said first enitters; 

computer means coupled to the first and second 
"radiation emitters and to the radiation sensors adapted to 
5 receive data from the sensors and from the emitters; 

means to calculate, based on the data received from 
said emitters and said sensors, the position and 
orientation of each radiation emitter with respect to said 
present time fixed coordinate system; 

computer means adapted to independently determine the 
positions and orientations of the first object and the 
second object relative to the present time fixed 
coordinate system, whereby determining the position and 
orientation of said first object relative to said second 
15 object, from the computed positions of the radiation 

emitters; 

computer means adapted to, at a frequency which is 
sufficient to substantially continuously accurately relate 
the present time position and orientation of the first 

20 object to the position and orientation of the model of the 

second object, relate the present time position and 
orientation of the second objecr with the previously 
obtained model of the second object and to electronically 
cross correlate the model of said second object and the 

25 present time position of said second object so as to form 
one composite image of said model which is consistent with 
the present tine position and orientation of said second 
object, and to superimpose the present time position of 
the first object on said consistent image in the present 

30 time coordinate system; and, preferably, display means 
coupled to the computer means adapted to display the 
location of the first object correctly superimposed on the 
model of the second object by correctly displaying a 
representation of the previously taken image of the second 

35 object which corresponds to the location and orientation 
of the second object in present time relative to the 
display of the first object in present time. 
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The method of this invention relates to the 
operation of the above described apparatus. This method 
will be described in relation to "seeing" the location of 
"a point of a surgical probe inside the cranium of a 
5 patient:, where the inside of the patient's cranium has 
been previously "seen" on an MRI or a CT scan. In 
carrying out this method, both the head of the patient, 
the second object according to this invention, and the 
surgical probe, the first object according to this 
10 invention, will be moved in illustration of the novel 
operation of this invention. 

The method of this invention, described relative 
ro the above described apparatus, includes the steps of: 
at a previous time, taking an MRI or a CT or the 
15 like, which may hereinbelow be referred to sometimes as 
the previous scan, through the patient's head with a 
sufficient number and location of slices as to reveal the 
internal structures of the patient/ s head. Where there is 
an abnormality, the number and location of the slices 
20 should be sufficient to depict that abnormality in at 

least one slice; 

establishing and storing an electronic file of the 
scan in the form of a model of the internals of the 
patient's head, including the abnormality, if there is 
25 one; 

relating that electronic file to a present time 
coordinate system; 

at thQ present time, as opposed to the previous time 
when the scan was originally taken, at sufficiently 

30 frequent intervals to accurately follow present time 
movement, detecting the positions of at least three of the 
radiation emitters operatively associated with the 
patient's head (the patient's head is the second object in 
the generic description of this invention) ; 

35 at the present time, at sufficiently frequent 

intervals to follow present time movement, computing, from 
the detecred positions of these emitters, the moving 
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locations and orientation of the second object relative to 
the predetemined fixed coordinate systezi; 

electronically adjusting the stored model to display 
"a view of that nodel which corresponds to the coinputed 
present time location and orientation of the moving second 
object in the same present time coordination system; 

at the present time, at sufficiently frequent 
intervals to follow present time movement, detecting the 
locations of at least two of the radiation emitters 
operatively associated with the probe (the probe is the 
first object in the generic description of this 
invention) ; 

at the present time, computing, from the detected 
positions of these emitters, at sufficiently frequent 
intervals to follow present time movement, the position 
and orientation of the first object in present time 
relative to the predetermined fixed coordinate system; 

at sufficiently frequent intervals to track the 
movement, of the first object relative to the second 
object, determining the positions and orientations of the 
first object relative to the second object by correlating 
the positions and orientations of the first object 
relative to the predetermined fixed coordinate system with 
the positions and orientations of the object relative to 
the sane predetemined fixed coordinate system; 

determining the correlation between the relative 
position and orientation of the probe with respect to the 
model of the object; and indicating the location of 

the probe with respect to the object by displaying a 
representation of the position and orientation of the 
probe in present time on the presently displayed model of 
the previously taken scan of the internal structures of 
the cranium. 

In the description of this invention, the 
radiation emitters have been located on the first and 
second objects, and the sensors for these radiations have 
been located in fixed positions within the present time 
coordination system, it is considered that this 
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arrangement of these enitters and sensors could readily be 
reversed. That is, the emitters could occupy the fixed 
positions in the present time coordination system and the 
"sensors could be located on the first" and second objects. 
5 The invention would operate in the same manner with either 
arrangenent of sensors and emitters. For convenience, the 
description of this invention has placed the emitters on 
the moving objects and the sensors in fixed positions. 
This arrangement should not be considered to be a 
10 limitation on either the apparatus or method of this 
invention. 



Brief. Description of the Drawings 

The accompanying drawing figures illustrate a 

preferred enbodioent of the present Invention and, 
15 together with the description, serve to explain the 

principles of the invention. 

Figure lA is a block flow diagram of the optical 

mensuration and correlation system of the present 

invention showing the major components of this system, 
20 except for the means to automatically measure the position 

and orientation of the movable and second object. 

Figure IB is similar to Fig. lA but includes the 

additional radiation emitters which will permit 

automatically measuring the position and orientation of 
25 the second object even during the movement of this second 

object. 

Figure 2 is a perspective view illustrating the 
invention in use by a surgeon performing intracranial 
surgery on a patient, and showing a cursor on a display 

3 0 screen that marks the corresponding position of the tip of 
the probe (the first object) within the image of 
previously obtained model data corresponding to the 
cranium (the second object) . 

Figure 3 is a view of a sample display showing 

35 a position of the tip of the probe superimposed on 
previously obtained model data of an inside slice of a 
cranium and the showing reference points of the second 
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object, aB depicted in figure lA, as triangles on the 
patient's skull. 

Figure 4 is a schena.tic perspective view of a 
sample of one of the one-dimensional photodetectors which 
5 are useful in the practice of the present invention. 

Figure 5 is a graph of the image intensity 
(manifested as a voltage or current) .versus locations on 
the photodetector surface for a typical light detector 
which could be used by the radiation supported mensuration 
10 and correlation apparatus of the present invention. 

Figures 6 and 7 are diagrams of the major steps 
performed by the computer to calculate the position of the 
probe (first object) with respect to the model of the 
inside of the cranium (the second object) and to display 
15 a cross-sectional image slice of the model of the inside 

of the cranium on a suitable display screen, such as a 
computer screen (CRT) . 

Figure 8 is a schematic view of radiation beams 
depicting an embodiment of this invention. 
20 Detailed Description of The Preferred Embodiments of This 

Invention 

The radiation mensuration and correlation 
apparatus 10 of the present invention, as applied to a 
medical application, which is illustrative of one use of 

25 this invention, is shown schenatically in Figure 1. It 

comprises a hand-held invasive probe 12 (first object) 
housing at least two radiation emitters 14 and 16 mounted 
collinear with one another and with the tip 13 of the 
probe 12. At 'least three remotely located, one- 

30 dimensional radiation sensors 20, 22 , and 24 are mounted 
in fixed, spaced relationship to each other and are 
located at known positions with respect to a predetermined 
fixed coordinate system 80 • The radiation sensors 20, 22, 
and 24 sense the radiation widely projected by the 

35 individual emitters 14 and 16 and generate electrical 

output signals from which are derived the location of the 
probe emitters 14 and 16 and, consequently the position 
and orientation of the probe tip 18 (which may not be 
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visible to the surgeon because it is within the cranial 
cavity) , with respect to the fixed coordinate system 80. 
In addition, where it is desired to determine the moving 
" position of the cranium during the operation, the three 
5 sensors 20, 22, and 24 can be prograniced to sense and 

derive the locations of other reference emitters 70, 72, 
and 7 4 on the second object 11 (Figure IB) in the same 
manner as for the probe emitters 14 and 16. The role of 
these reference emitters on the second object is to 
10 automate the calculation of the relationships between the 
present time coordinate system of the model's image 13 
(Figure 2) of the second object, the coordinate system of 
the sensors, and the local coordinate sysrem of the second 
object itself Il- 
ls A control unit 30 connected to the moveable 
probe 12 via a data line 2 6 and coupled to the remotely 
located sensors 20, 22, and 24 via data lines 28, 32, and 
34, respectively, synchronizes the sensing of the five 
(exemplary) emitters and the differentiation between them. 
2 0 In that eirliodiment of this invention where the various 
emitters emit radiation in pulses, as by strobing them, 
the control unit is adapted to control the time 
multiplexing of the two emitters 14 and 16 on the probe 
and the three emitters 70, 72 and 74 on the cranium, 
25 controls the operation of the sensors 20, 22, and 24, and 
receives dif f erentiatable data from these sensors as will 
be more completely described below, A coordinate computer 
36, coupled to ths control unit 30 by a data line 38, 
calculates the three-dimensional spatial location of the 
30 probe emitters 14 end 16 and consequently the position and 
orientation of the probe tip 18, and correlates those 
positions with data from correlation information 42 and 
from a model 13 of the second object 11 which has been 
previously stored electronically in an electronically 
35 accessible database 40. Finally, the computer 36 causes 
an associated* cathode ray tube-monitor (CRT) to display 
the representation of the position and the orientation of 
the probe tip 18 with respect to the computer image 13 of 
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the craniuio 11 on display screen 44 (Figure 2) as will be 
more fully described below. 

The probe 12 could be used without the cable 26, 
"in that it could be coupled to the control unit 3 0 by 
5 employing distinctive modulation of the light emitters 14 

and 16 instead of sequentially energizing (strobing) then, 
or by varying the wavelength or type of the radiation 
emitted therefrom- For example, the wave forms, color, or 
frequencies of each could be different. In the case of 

10 using sound radiation, the frequencies of the sound 
emitted by the different emitters could be varied so as to 
differentiate between them. The controller 30, by 
detecting the differences between different emitters, that 
is- the wave form, color, frequency or other dissimilarity, 

15 of the emitted radiation, can determine to which emitter 

the sensors 20, 22, and 24 are reacting - 

The fundamental mensuration and correlation 
apparatus 10 of the present invention has been illustrated 
in connection with aiding surgeons performing delicate 

20 intracranial surgery. This general use of this apparatus 

does not constitute this invention, nor is it a limitation 
thereon. This use will only serve to illustrate this 
invention. The remaining description continues to use 
such a surgical embodiment as illustrative, although many 

25 other surgical or other applications besides intra-cranial 
surgery are possible (for example, back or sinus surgery 
and breast biopsy). Moreover, the radiation mensuration 
and correlation apparatus 10 of this invention may be used 
for other purposes in many various medical or non-medical 

30 fields. In the described embodiment, the physical object 

11 of interest, that is the second object in the generic 
application of this invention, is the head or cranium of 
a patient, and the model of the cranium is replicated 
using a series of parallel internal image slices (of )cnown 

35 mutual spatial relationship) such as those obtained by 
means of computed tomography (CT) or nuclear magnetic 
resonance imaging (MRI) . These i:nage slices are then 
digitized, forming a three-dimensional computer model of 
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the patient's craniurn which is then stored in the 
electronically accessible database 40. 

As shown in Figures lA, IB, 2, and 3 a surgeon 
■places the tip 18 of the probe 12/" that is the first 
5 object, at any point on or inside the cranium 11 of the 
patient. The position sensors 20, 22, and 24 detect the 
locations of the emitters 14 and 16 attached to the 
portion of the probe 12 that remains outside the patient's 
body. In order zo accomplish this, the radiation produced 

10 by the emitters 14 and 16 must be "visible" to the sensors 
20, 22, and 24. For that reason, more than two emitters 
nay be placed on the first object so that the radiation 
from at least two of them will always be visible to the 
sensors. These emitters 14 and 16 are effectively point 

15 sources and radiate energy through a wide angle so that 
this radiation is visible at the sensors over a wide range 
of probe orientations and positions. 

The sensors 20, 22, and 24, the control unit 30, 
and the computer 3 6 cooperate to determine the three- 

20 dimensional location of each emitter 14 and 16 within a 
coordinate system, and compute the coordinates of each 
emitter in the predetermined fixed coordinate system 80, 
in present time. The computer 3 6 can then calculate the 
position and orientation of the tip IS of the probe 12 

25 with respect to the predetermined fixed coordinate system 
80, according to the locations of the emitters within the 
fixed coordinate system 80 and the dimensions of the 
probe, which dimensions had been placed into the memory 
(not shown) of the computer 36 beforehand. It should be 

30 noticed that the computer 36 can also easily compute 
position and orientation information about other specific 
locations on the probe (such as the vector from emitter 14 
to the tip IB)* Once the computer 36 has calculated the 
location of the probe tip 18 with respect to the fixed 

35 coordinate system 80, the computer 36 then uses the 
relationship between the model of the cranium, which had 
previously been obtained and stored in the database 40, 
and the f ixed_coordinate system 80 to calculate the 
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position and orientation of the probe tip 18 in relation 
to the model of the second object 11. Finally, the 
computer 36 displays a representation of the inodel- 
"relative position and the orientation of the tip 18 on a 
5 display screen 44. In a simple form of the preferred 
embodiment of this invention, the computer 36 accomplishes 
this, display by accessing a previously taken CT or HRI 
image slice 13 stored in the database 4 0 that is closest 
to the present time position of the probe tip 18, and then 

10 superimposes a suitable representation 76 of the tip 18 on 

the image 13 as shovn in Figures 2 and 3.. Thus, the 
surgeon knows the precise position and orientation of the 
tip 18 in the patient's cranium relative to the image data 
by merely observing the display screen 44. A most 

15 preferred form of the present invention can derive and 

display an arbitrar\' oblique cross-section through the 
multiple image slices of the MRI, etc, where the cross- 
section can be, for example, perpendicular to the probe 
orientation. 

20 The details of the optical mensuration and 

correlation apparatus 10 of the present invention are best 
understood by reference to Figures 1 and 4 collectively. 
Essentially, the probe 12 supports the two radiation 
emitters 14 and 16, which are rigidly attached to the 

2 5 probe 12 at fixed, known distances from each other as well 

as from the probe tip. Since only two emitters are used 
here as being representative of the practice of this 
invention, the emitters 14 and 16 should preferably be 
collinear with the tip 18 of the probe 12 so that the 
30 computer 36 can determine uniquely the position and 
orientation of the tip 18 in three dimensions. Moreover, 
for reasonable measurement accuracy, the emitters 14 and 
16 should preferably be. at least as far from each other as 
the nearest one is from the tip 18. In any case, the 

3 5 geometrical relationship of the emitters 14 and 16 to each 

other and to the probe tip 18 must be specified to the 
com.puter 3 6 beforehand so that the computer 36 can compute 
the exact location of the tip 18 based on the locations of 
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the individual radiation emitters 14 and 16. The use of 
three or more ncn-collinear emitters would not require 
that any two of tlieru to be collinear with the probe tip. 
"Three or Tiore non-collinear emitters would pennit the 
5 coTiiputer to coaipute full position and orientation 
information (yaw, pitch, and roll) for the probe. 

^Although the invention is described as showing only a 

cursor locating the relative position of the probe tip 18, 
the invention can be modified to display a line or a 
10 shaped graphic or other icon to indicate the position and 
orientation of the probe 12- This would entail only the 
determination of additional points on the probe in the 
saae way that the tip of the probe is located. 

The two radiation emitters 14 and 16, as well as 
15 the additional radiation emitters 70, 12, and 14, can be, 
and preferably are, high intensity light enitting diodes 
(LSDs) , which are preferably coordinated such that the 
e:nission for any one source is distinguishable fron the 
emissions from the other sources. One such 

20 differentiation is to have the emitters time-multiplexed 

or stirobed by the control unit 30 in- a predetermined 
sequence such that only one light emitter is '»on" or 
emitting light at any one time. The light emitted from 
any one of these enitters is detected by each of the three 
25 light sensors 20, 22, and 24, which then determines the 
location of each particular emitter in relation to the 
known positions of the sensors 20, 22, and 24 at the time 
it is strobed. 

Each of the one-dimensional sensors 20, 22, and 
3 0 24 used in the preferred embodiment 10 of the present 
invention can be identical to the others in every respect. 
Therefore, for the purpose of giving a detailed 
description of this embodiment, only the sensor 20 is 
shown and described in detail in figure 4 since the 
35 remaining sensors 22 and 24 are identical. 

In Figure 4, the representative one-dimensional 
sensor 20 comprises a cylindrical lens 46 having a 
longitudinal axis 48 which is orthogonal to the optical 
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axis 50 of the sensor 20. A linear radiation detector 52, 
such as a charge coupled device (CCD) with several 
thousand elements (or a similar device capable of linear 
"positional radiation detection of a suitable "image"), is 
5 positioned in such a manner that the "optical" axis 50 
passes through the center of the aperture 54 of the 

- ra diatio n detector 52 an d such that the longitudinal axis 

of the aperture 54 is orthogonal to the longitudinal axis 
48 of the lens 46. Radiation, such as light beams, 56 

10 from the emitters 14 (and in the same manner emitters 16, 
70, 72, and/or 74) are focused by the cylindrical lens 46 
into a real image line 58 on the surface 60 of linear 
detector 52. 

The detector, illustrated by a photodetector 52, 

15 then generates an output 68 (Figure 5) that is related to 

the position of a real image line 58 on the surface 60 of 
photodetector 52, thus characterizing the location of the 
image itself. That is, those elements or points of the 
photodetector 52 illuminated by the real image line 58 

20 will generate a strong signal, while those not illuminated 
will generate none or very weak signals. Thus, a graph of 
image intensity (or signal strength) versus locations on 
the surface of the photodetector will resemble a signal 
peak curve 68 (see for example Figure 5) . The "all- 

25 emitters-of f *' (or background) signal level 66 is never 

quite zero due to the effects of environmental radiation, 
such as light in the operating room, electronic noise, and 
imperfections in the photodetector. In any event, since 
the image of the illuminated emitter is focused into line 

3 0 58, only the angular displacement of emitter 14 from the 
optical axis 50 in the plane of the longitudinal sensor 
axis 54 is measured by the sensor 52, hence the 
designation "one-dimensional sensor". 

Thus, a single one-dimensional sensor 20 can 

3 5 only locate the plane on which a radiating emitter 14 
lies. The detector 20 cannot, by itself, determine the 
unique point in space on that plane at which radiating 
emitter 14 is located. To precisely determine the 
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locawion in space of the radiating emitter 14 requires at 
least three such sensors positioned in spaced relationship 
to each other, since the intersection of three planes 
"defined by the three sensors, respectively, are "required 
5 to define a single point in space. 

To locate the position of one particular 
radiating emitter, such as 14, the sensors 20, 22, and 24 
are nounted so that the optical axes of their lenses 48 
are not all parallel and no two of such axes are 

10 collinear. In a preferred embodiment of this invention, 
two light sensors, such as sensors 20 and 24 in Figure 2, 
are situated so that their respective axes 48 (Figure 4) 
are in parallel, spaced relationship, and the third 
detector 22 is situated between and equidistant from the 

15 other two detectors, but with its axis 4 8 perpendicular to 
the axes of the other two. That is, the sensors 20, 22, 
and 24 should be arranged along a line or arc (Figure 2), 
such that each sensor 20, 22, and 24 is generally 
equidistant from the center of the volume in which the 

20 measurements are made, equally spaced from each other, and 
all aimed at the center of the measurement volume. 
Suppose for example that the sensors 20, 22, and 24 are 
arranged along a horizontal arc and the optical axes of 
all sensors are oriented horizontally. Then the middle 

25 sensor should be oriented so as to measure the angular 
elevation of the radiation emitters as described above. 
The two outer sensors measure the horizontal angle 
(azimuth) relative to the fixed coordinate system 50. 
Data from the outer sensors are used to stereographically 

30 calculate both the horizontal position and distance from 
the sensors as will be more fully described below. 

The accuracy of three-dimensional measurement 
depends on the angle formed between the optical axes of 
the outer two sensors 20 and 24, where the emitter to be 

3 5 measured is at the vertex of the angle. Accuracy will 
improve" as that angle approaches a right angle. At least 
three of the several possible sensors 20, 22, and 24 must 
be spaced so that the desired measurement volume is 
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completely within their field of view which can be 
accomplished by making the fecal length of the lens 46 
short enough to provide coverage of the entire desired 
" field of view. In another embodinen't of this invention, 
5 additional sensors, which nay be substantially identical 

to sensors 20, 22, and 24, could be used to provide more 
viewpoints, to broaden coverage of the field of view, or 
to enhance measurement accuracy. 

While this process of detecting a given 
10 radiating emitter, such as 14, can determine the exact 
location of the radiating emitter, it cannot by itself 
determine the particular orientation and position of the 
probe or its tip 16 in three-dimensional space. To do so 
with only two emitters recruires that both emitters 14 and 
15 16 be collinear with the probe tip 18, as described above. 

Also, the distances between each emitter, 14 and 16, and 
the probe tip (as well as the distances between the 
emitters 14 and 16 themselves) must be known and loaded 
into the memory of the computer 36 before the computer 3 6 
20 can determine the position and orientation of the probe 
tip 18 from the locations of the emitters 14 and 16 in the 
fixed coordinate system 80. Consequently, when each of 
the radiation emitters 14 and 16 is rapidly turned on in 
sequence, or strobed, the sensors 20, 22, and 24. can 
25 detect the exact location of each emitter in turn. Thus 
computer 36 can determine the exact position and 
orientation of the probe, and therefore its tip 18. Since 
only one of the radiation emitters 14 or 16 is on at any 
one time, the detectors 20, 22, and 24 locate the location 
30 of that particular illuminated emitter individually. If 
the strobe rate, that is, the frequency at which the 
emitters 14 and 16 are turned on and off in , sequence, is 
fast enough, the detectors 20, 22, and 24 can, for all 
practical purposes, determine the position and orientation 
35 of the probe 12 and its tip 18 at any instant in time, and 
therefore can follow the movement of the probe tip in 
present time, that is during the time that the probe tip 
is actually moving. In other words, this system can 
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simulate the nover>ent of the probe tip on the previously 
taken iniage in present tine during the surgical procedure. 
The sensors 20, 22, and 24 need only distinguish 
"which of the radiation emitters 14, 16, 70, 72, or 74 is 
5 on at any one time. In the preferred embodiment 10 of the 
present invention, this function is accomplished by 
strobing each of the emitters in sequence, as described 
above. However, other nethods can be used to allow the 
sensors 20, 22, and 24 to distinguish the respective 

10 radiation emitters 14, 16, 70, 72, and 74 from one 
another. For example, different wave lengths (colors) of 
light, or different frequencies of sound, could be used in 
conjunction with detectors capable of distinguishing those 
particular different radiations. 

15 Alternatively, it is one aspect of this 

invention to modulate each of the respective radiation 
emitters 14, 16, 70, 72, and 74 with a unique wave form or 
pulse train. This means of differentiating between the 
different emitters is believed to be novel and unique to 

20 the instant invention. If such different wave forms or 

pulse trains are used to differentiate the different 
emitters, it is also within the spirit and scope of this 
invention to transmit additional information on these wave 
forms, such as for example the temperature of the 

25 particular structure being contacted by the probe tip 18. 

It is within the scope of this invention to provide means 
readily accessible to the surgeon or other operator to 
engage or disengage the taking of such additional 
information and the transmission thereof by the unique 

30 wave forms or pulse trains radiated by the respective 
emitters. Under these circumstances, the control unit 

30 or computer 36 will be designed to demodulate the wave 
form to determine to which particular emitter the sensed 
signal belongs, and to decade the additional information 

35 being transmitted. 

Numerous other methods for distinguishing the 
radiation emitters are possible. Therefore, the present 
invention should not be regarded as limited to the 
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particular strobing method shown and described herein, but 
is generic to the use of any neans to differentiate 
between the different emitters. 

Conventional or unique auto-focusing or 
5 multiple-lens radiation detection nay be integrated into 
the sensors 20, 22, and 24 to improve the performance of 
the system. However, the simple, fixed-focus optics shown 
and described herein and shown in Figure 4 for one sensor 
provide a good level of performance if the working range 

10 of the probe is restricted* Even if the real image of an 

emitter, such as 14, is somewhat out of focus on the 
detector 52, the angular measurement of the image is still 
usable. A usable measurement for each of the sensors 20, 
22, or 24 to generate will be any of the following: (1) 

15 the position of the detector element with peak intensity, 

(2) the intensity-weighted average (centroid) of all over- 
threshold elements, or simply (3) the average of the 
minimum and maximum elements where the intensity is over 
some threshold. The detector 52 should be placed at the 

20 focal distance for the farthest typical operating distance 

of the radiation emitters. Closer emitters will form 
slightly defocused images 58, but they require less 
precise angxilar measurement for a given distance accuracy. 
Furthermore, their de-focused real images are brighter, 

25 which increases the brightness gradient at the edges of 
the image. 

As described so far, the real image 58 of the 
currently activated emitter roust be significantly 
different from (for example brighter than) the rest of the 

30 radiation falling on the sensor 52. Otherwise, other 
lights or reflective surfaces in the field of view of the 
sensors will hinder the detection of the emitter's real 
image. Therefore, it is desirable to include in the 
apparatus, circuitry to subtract the background radiation 

35 received by the sensors from other, ambient, sources. 

This per se known circuitry enhances use of the 
invention where the sensors are required to detect the 
radiation emitters against relatively bright backgrounds. 
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While the radiation emitters are all nomentarily 
extinguished, the one^dimensional data froin each sensor 
are saved in a laemory. This can be done in an analog 
"delay line or by digitally sampling the output signal and 
5 storing it in a digital menory. Then, as each emitter is 

"viewed" sequentially, the saved data are subtracted fron 
the current data generated by the currently radiating 
emitter. If the background data are stored digitally, the 
current data are also digitized, and the stored background 

10 data are digitally subtracted from the current data. 

A graphical representation of the radiation 
intensity of the inage or, equivalently , the generated 
output voltage amplitude for each element in a row of 
detecting elements, is shown in Figure 5. The graph 

15 depicts typical background image intensities 66 with all 
emitters off, the intensities 68 with one radiation 
emitter on, and the element-by-element difference 64 
between the intensities with the emitter off and those 
with it on. The measurements will likely contain some 

20 random noise, electronic or otherwise, and two consecutive 

measurements for a given sensor element may differ 
slightly even where the background is unchanged. 
Therefore, the differential intensities 64 between two 
consecutive measurements also contain some random 

25 electronic noise. However, the two measurements differ 
substantially only at the location of the radiation 
emitter image, and this difference exceeds the threshold 
level 62. 

The details of the structure and operation of 
30 the control unit 30 . are best seen in Figure 6. 

Specifically, control unit 3 0 (see Figure lA and IB) 
supplies power to the radiation emitters 14, 16, 70, 72, 
and 74 and the radiation sensors 20, 22, and 24. A 
control and synchronization unit 84 and radiation source 
35 sequencer 88 (where a strobed radiation sequencing is 

used) time-multiplexes or strobes the radiation emitters 
individually, as described above, so that the position and 
orientation of the probe tip 18 (Figure 1) can be 
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determined from the signals received fron the sensors 20, 
22, and 24. The angular data signals received fron the 
sensors 20, 22, and 24 are converted by an analog-to- 
-digital converter 92. Actually, three analog-to-digital 
5 converters are used, as shown in Figure 6, but only one is 
labeled and described herein for brevity, since the other 
two analog-to-digital converters are substantially 
identical and are used to convert the signals from the 
other sensors 22 anii 24. 

10 The control and synchronization unit 84 also 

controls three switches, of which switch 93 is typical, 
which store all digital data received fron the sensors 20, 
22, and 24 when the radiation emitters 14 and 16 are off 
and stores rhese data into a background rie:nory 94 • Then, 

15 when the radiation emitters 14, 16, 70, 72, and 74 are 

illuminated in sequence by radiation source sequencer 18, 
the synchronization and control unit 84 changes the state 
of switch 93 which then redirects the data from the three 
sensors 20, 22, and 24 to a subtraction unit 91. The 

20 subtraction unit 91 subtracts the background data from the 

emitter radiation data, thus resulting in a signal which 
has been relatively freed from the background signal 66 
(Figure 5) since the fixed pattern noise has been 
subtracted from the signal. 

25 As shown in Figure 6, which should be considered 

in conjunction with Figure 5, a 1-D (one-dimensional) 
position calculation unit 95 determines the location of 
the real image line 58 on the CCD sensor 52 (Figure 4) by 
measuring the locations of the edges 67 and 69 of the 

30 signal blip 68 (Figure 5) generated by the CCD sensor 
based on a predetermined threshold signal level 62. The 
1-D position calculation unit 95 then averages the 
distance between the two edges to find the center of the 
signal peak 68 as shown in Figure 5. This method of 

35 determining the center of the signal peak is per se well 

known in the" art and need not be described in further 
detail. Moreover, numerous other methods of determining 
the location of the signal peak or its centroid are known 
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in the art and will be obvious to those of ordinary skill 
in the art. The method used depends on the signal 
characteristics of the radiation sensor used as well as 
*the characteristics of the lens system used to focus the 
5 radiation onto the surface of the detector, in addition to 
other paraineters. Those practicing this invention with 
the various alternatives described herein would have no 
trouble selecting a signal detection algorithm best suited 
to the particular characteristics of the sensors and the 

10 particular radiation being used. 

Finally, the control unit 3 0 (Figure 1) 
transmits the radiation data to the computer 36. That is, 
when the computer 36 is ready to compute the current 
location of the currently radiating enitter, such as 14, 

15 the latest angular data from all sensors 20, 22, and 24 
are provided for analysis. If the sensors generate data 
faster than the control unit 3 0 can process then, the 
surplus angular data are simply discarded. 

The operation of the computer 3 6 is most 

20 advantageously set forth in Figure 7, The computer 3 6 

calculates one-dinensional positions for each radiation 
emitter such as 14 or 16, based on the location of the 
signal peak from each respective sensor 20, 22, and 24. 
These one-dimensional angular position measurements are 

25 then used to determine the three-dimensional spatial 
coordinates of the emitters 14 and 16 and thus for the 
position and orientation of the probe 12 relative to the 
predetermined fixed coordinate system 80 by coordinate 
transformation methods which are per se well-known in the 

30 art. The output signals from the computer 36 can be in 
any form desired by the operator or required by the 
application system, such as XYZ coordinate triples based 
upon the predetermined fixed coordinate system 80. 

Figure 8 and the following paragraphs describe 

35 in detail how the location of a single radiation emitter, 
such as 14, is computed from the data derived from the 
sensors 20, 22, and 24. The following description applies 
to these three sensors 20, 22, and 24 only. If there are 
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more than three such sensors, the calculation can be 
performed using any three or more of the sensors. 
Furthemore; if more than three sensors are used, the 
-average of the points calculated from ''all combinations of 
5 three sensors could be used to increase accuracy. Another 

option is to use the point calculated from the three 
sensors closest to the radiation emitter 14 or 16. The 
following parameters are considered to be known XY2 
constants : 

10 DO[i], one endpoint of each linear photodetector i; 

Dl[i], the other endpoint of linear photodetector i; 
LO[i], one endpoint of the axis of each lens i; and 
Ll[i), the other endpoint of the axis of lens i. 
Each sensor generates T[i], a parametric value between 0 
15 and 1 indicating where the peak or center of the line 

image of the emitter intersects the line segment between 
DO[i] and Dl[i]. The XYZ coordinates of point S are to be 
calculated, where S is the location of the radiation 
emitter. For a CCD radiation detector array, T[i] is the 
20 index of the element on which the center or peak of the 

image falls divided by the number of elements on the 
detector array. 

The three-dimensional coordinates of the above 
points are all referenced to a predetermined fixed 

2 5 coordinate system 80. The cylindrical lens and linear 

photodetector do not directly measure the angle A of the 
radiation emitter about its lens axis; rather, they 
measure a value T[i] linearly related to the tangent of 
that angle: 

30 tan(A) = C * (2 * T[i] - 1), 

where C is a constant of proportionality that is related 
to, and determined empirically by, the dimensions of a 
particular system. 

The three-dimensional location of the image line 

3 5 on the linear photodetector is: 
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D[i) = (1 - T[il) * DO[i] - (T[i]) * Dl[i] 

If the Isns is ideal, then S also lies in plane 
-p[i]. In reality, the point D(i]' 'might have to be 
computed by a non-linear function F(t) that corrects for 
5 non-linear aberrations of the lens or the photodetector: 

D[i] = (1 - F(T[i])) * DO[i] + (F(T[i])) * Dl[i) 

Function F(t) could be a polynomial in variable 
T, or it could be a value interpolated from an empirically 
detemined table. 

10 P[i] is the unique plane determined by the three 

points D[i), LO(i], and Ll[i], which are never col linear. 
S is the point of intersection of the planes P(l], P[2], 
and P[3] determined respectively by sensors 1, 2, and 3. 
S is a unique point if at least two sensor lenses 

15 longitudinal axes 48 are no^ parallel and if no two lens 

axes 48 are collinear. The intersection point is found by 
finding the corjiion solution S of the three equations 
defining the planes Pti]. Once the location S of each of 
the probe's radiation emitters is computed, the location 

20 of the probe's tip 18 can be calculated- The method of 

making such a determination is well known using the 
teaching of analytic geometry and matrix manipulations. 

If M is a linear transformation describing the 
relationship between a point R in the image coordinate 

25 system and a point S in the fixed coordinate system, then: 

R * H = S. 

If M'** is the inverse of M and if S is a point in the 
fixed coordinate system, then the point R in the image 
coordinate system corresponding to S is: 

30 • S * H'' = R. 

Now, suppose that the second object is moved in the 
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mensuration coordinate system. This can be described by 
o linear transformation U where the coordinates S of a 
point are mapped zo the coordinates S': 

S * U = S' 

5 Then the old value of M above must be multiplied by U in 
order to correct the relationship between the point R in 
the image coordinate and the corresponding point in the 
mensuration coordinate system because of the relative 
movement of first object with respect to the second 
10 object: 

R = S' * u"' * M"' 

The preliminary steps required before practicing the 
method of the invention are now described. Then, after 
fully describing these preliminary steps, the detailed 
15 steps of the method of the optical mensuration and 

correlation apparatus are described. 

Use of the invention takes place in three 
phases: tJie imaging phase, the correlation phase, and the 
normal operation phase. The imaging phase precedes zhe 

2 0 normal operation of the present invention. During the 

imaging phase, a scan of the body of the second object of 
interest is used to build a three-dimensional geometrical 
model. In the preceding description, the second object 
was the head of a human intracranial surgical patient 
25 because the invention is advantageously used in 
stereotactic neurosurgery. Accordingly, the three- 
dimensional model comprises digital data from a series of 
internal cross-sectional images obtained from computed 
tomography (CT) , magnetic resonance (MRI) , ultrasound, or 

3 0 some other diagnostic medical scanner. In any case, the 

image data are stored in a suitable, electronic memory 4 0 
which can be accessed later by the computer 36. The data 
are considered to be stored as a series of parallel two- 
dimensional rectangular arrays of picture elements 
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(pixels), each pixel being an integer representing, 
relative density. If the object is relatively rigid, like 
a hunan head, this three-dimensional model nay be created 
-at some time before the correlation arid operational phases 
5 of the invention and possibly at another location. 

Also, during the imaging phase, at least three 
non-collinear reference points 71, 73, and 75 (Figures 2 
and 3) must be identified relative to the object 11. 
These may be represented by ink spots, tattoos, radiopaque 

10 beads, well-defined rigid anatomical landmarks, locations 

on a stereotactic frame, sterile pins temporarily inserted 
into rigid tissue or bone of a surgical patient, or some 
other reference means. The coordinates of these reference 
points are neasured and recorded relative to the 

15 coordinate system of the imaging device. One way to 

acconplish this is to capture the reference points as part 
of the previously roade three dimensional model itself. 
For example, radiopaque pins could be placed within the 
image planes of diagnostic CT slices; the pin locations, 

20 if not automatically detectable from their high density, 
can be identified interactively by the surgeon using a 
cursor on the computer display of the CT slices. See 
Figure 3 . 

The initializing of the position and the 
25 orientation of the second object, the patient's cranium, 
is well )mown in this art. The instant invention departs 
from this well known operation to add radiation emitters 
which have a known and exact spacial relation to these 
fiducial markings. These additional radiation emitters 
30 must then be programmed or otherwise activated for use in 

a particular manner in order to practice the instant 
invention. They must be programmed to radiate at some 
frequency during the surgical procedure which is in 
progress during present time so that the position and 
3 5 orientation of the second object will be available to the 

surgeon- at all relevant times and so that this position 
and orientation can be repeatedly and automatically 
updated in order for the system to revise the specific 
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selected scan slice to superirrioose the position and the 
orientation of the first object on in a correct 
depictation of the actual relative positions and 
-orientations of both the first and second object in 
5 present time during the surgical procedure. 

The initial correlation node itunediately 
precedes the normal operational phase of the present 
invention and must take place in the operating room. 
During this initial correlation phase, the instant system 

10 accesses the data of the three-dimensional geometrical 
model of the patient (or other object) , including the 
reference point (fiducial marker) coordinates which were 
recorded earlier, that is previously. Next, the surgeon 
may place the tip of the probe 18 ar each of the reference 

15 points 71, 73, and 75 on the patient, in turn. This 
sequence of operations may be directed by the computer 
program. In the alternative, the sysreia of this invention 
provides these data automatically by the radiation from 
the emitters 70, 72, and 74 being received by the sensors 

20 directly and automatically without special intervention by 
the surgeon. Either of these procedures estciblish an 
initial relationship between the locations of these 
reference points in the model coordinate system and their 
current physical locations in the fixed coordinate system 

25 80. However, the preferred determination of this initial 
position and orientation also carries on during the whole 
of the surgical procedure and therefore is capable of 
substantially continuously updating the position and 
orientation of the second object and relating it to the 

30 current position and orientation of the probe. In turn, 
this establishes a linear mathematical relationship 
between all points in the model and points in the 
coordinate system 80. Thereafter, when the patient is 
moved relative to the sensors, the prior art must 

35 establish a new relationship by again digitizing the 
reference points 71, 73, and 75 within the coordinate 
system 80. That is, the correlation phase must be 
repeated. Again, the system of this invention uses the 
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emitters 70, 72 and 74 to accomplish this auto:natically • 
For this reason, the autoraatic tracking of the position of 
the head, or the second object whatever that is, which is 
-described below and which overcomes 'this problem, is an 
essential, significant feature of the present invention. 
Since the position and orientation of the head 
is initially and substantially continually thereafter 
correlated with the model", the" surgeon ' can relate any 
locations of interest on the diagnostic images with the 
corresponding physical locations on this patient during 
the operation, and vice versa. These include locations 
accessible to the probe tip 18 but not necessarily 
directly visible to the surgeon. 

Having described the function and purpose of the 
prelininary steps, the detailed method of the present 
invention is more easily understood. As shown in Figure 
7, the position data 21 of the probe emitters generated by 
the sensors and control unit are converted into three- 
dimensional coordinates relative to the predetermined 
fixed coordinate system 80 of the sensors. Using 
dimensional parameters describing the relationship among 
the probe emitters and the probe tip, the computer 
determines the coordinates of the probe tip in a step 39. 
During the initial correlation phase, the probe tip may be 
placed at each of the reference points 71, 73, and 75 in 
turn. Alternatively, in accord with a preferred aspect of 
this invention, the emitters 71, 73 and 75 are located by 
the sensors and the correct position and orientation of 
the second object is thereby determined. The coordinates 
of the second object in the fixed coordinate system along 
with their coordinates 4 6 in the image coordinate system 
determine a unique linear transformation relating the two 
coordinate systems in a step 45. This is a per se known 
calculation in analytic geometry and matrix mathematics. 

As noted above, a more automated and direct 
method of determining the location of the second object is 
to directly read the locations of the fiducial points 71, 
73 and 75 by the fixed sensors 20, 22 and 24. This can be 
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accomplished by placing radiation emitters 70, 72, and 74 
(Figure IB) at those reference points (or in a known fixed 
spacial relationship zo those reference points 71, 73, and 
-75) . ' The emissions of these emitters can then be read 
5 directly by the sensors 20, 22, and 24, and thus the 

computer can then automatically determine their locations 
relative to the predetermined fixed coordinate system 80 
of the sensors. Thus the position and orientation of the 
second object, the cranium in the preferred embodiment of 

10 this invention, can be automatically and substantially 
continuously determined. With the position and 
orientation of the second object being at least 
frequently, if not substantially continuously, updated, 
the position of the first object, the probe, which is also 

15 determined at least frequently, if not substantially 
continuously, can then be updated in relation to the 
second object at the same frequency. Since the position 
and orientation of both the first and the second objects 
are each at least frequently determined and updated in 

20 relation to the fixed coordinate system 80, the position 

and orientation of each of these first and second objects 
can then be determined relative zo each other, .by 
indirect, but well known, calculations which are easily 
carried out in short order by a computer. 

25 It has been stated herein that the position and 

orientation of the second object, the cranium, can, 
according to this invention, be determined continuously or 
at least frequently. The frequency at which the position 
and orientation of the second object is determined is a 

30 function of the desires of the operator of this system and 

the frequency at which the radiation emitters and the 
sensors can be operated. In the case where the emitters 
all emit the same wave length of radiation and the sensors 
all sense this same wave length of radiation, the 

35 differentiation of the emissions of the several emitters 
is followed in a sequential pattern- Thus, in this 
embodiment of this invention, the emitters will emit 
radiation in sequence, for example 14, then 16, then 70, 
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then 72 and then 74. The sensors will have been 
programned to identify a signal with an emitter as a 
function of when the signal is received. 

In this eTr±)odiment of this invention, the 
5 position and orientation of the first object, the probe, 

is detennined with the same frequency as is the location 
and the orientation of the second object, the cranium, 
because all of the emitters radiate in sequence. However, 
the system can be programmed so that the emitters 14 and 

10 16 fire more or less frequently than the emitters 70, 72 
and 74. Under these conditions, the position and 
orientation of the first object and of the second object 
will be detemined at different individual frequencies, 
that is at the same frequency as the frequency of the 

15 -radiation from their respective emitters. It will 
therefore be clear that the frequency of determination of 
the location of any given emitter, and therefore the 
determination of the position and orientation of these 
first and second objects, is, because of the instant 

20 invented system, for the first tine entirely controllable 

by the programr.er or the operator, within. the capabilities 
of the system operating the emitters. 

However, it should be understood that the 
position and orientation of the first and/or second 

25 objects can be detemined in a substantially continuous 

manner. In this embodiment of this invention, each 
emitter will radiate a different wave length or wave form 
or frequency pulse of radiation. Therefore, the radiation 
emitted from each emitter is simultaneously distinct from 

3 0 the radiation emitted from the other emitters. Under 
these conditions, the location of each emitter can be 
determined continuously by a set of sensors which is tuned 
to the specific, different radiation of each emitter. 
Therefore, the location of each emitter can be determined 

35 continuously, whereby the position and orientation of 
either or both of the objects can be calculated by the 
computer from these continuous locations of the different 
emitters. 
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While it is a significant distinction of this 
invention fron the prior art that: 
in the prior art: 

the first object is intended to be moved and the 
5 second object is intended to be stationary; and h 
position and orientation of the first object is frequently 
determined, but the position and orientation of the second 
object is only determined at the start of the operation 
and at any time that the second object, the cranium, which 
10 is intended not to be moved at all during the operation, 
is known by the surgeon to be moved; 
whereas 

according to this invention: 

the first object is intended to be moved, and 
15 the second object is nor intended to be rigidly 
immobilized in place, or, put another way, the second 
object is permitted to move and is even expected to move; 
and 

the position and the orientation of the first 

20 object is frequently determined, and the position and 
orientation of the second object is also frequently 
determined. The position and orientation of these two 
objects may be determined at the same frequency or at 
different frequencies (or even continuously) as desired by 

25 the operator. 

In preferred embodiments of this invention, the 
position and ori-entation of the second object will be 
deteinnined from one hundredth to ten times, most 
preferably from a quarter as often to four tines, as often 

30 as the frequency at which the position and orientation of 
the first object is determined. As a general proposition, 
there is no limit on the relationship between these 
frequencies of measurement. The preferred relationships 
set forth herein are illustrative and not limiting. The 

35 frequency of each measurement is dependent on the amount 

of movement which is allowed and is intended to be shown 
on the CRT. The upper limit on this frequency is 
determined by the ability of the emitters to be 
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distinguished. There is no lower linit. 

In the instanr specification, the emitters have 
been described as being on the first and second objects 
-and being movable therewith, and the sensors have been 
5 described as being in a fixed relation to the coordinate 

system. While this is the preferred system, it is by no 
means the only configuration of the system of this 
invention. It is also within the scope of this invention 
to provide the eniitters in fixed relationship to the 

10 coordinate system, and the sensors on the first and second 
objects, respectively. The wiring may be somewhat more 
cumbersome in this configuration, but that should not 
detract from the viability of such a reversal. 

This invention has been described with reference 

15 to a first and a second noving object, and the 

determination of each of their absolute and relative 
positions and orientations in a fixed coordinate system. 
It will be clear that this same system applies to more 
than two objects. In fact, the position and orientation 

20 of any number of objects can be determined, both 

absolutely with respect to the coordination system, and 
relatively with respect to each other, by the practice of 
this invention. Thus, when this specification and the 
claims appended hereto speak of a first and a second 

25 object, these can be two out of any number of total 

objects- This number is merely illustrative of the 
practice of this invenrion and is in no way liiaiting 
thereon. 

Thus, the preferred system of this invention 
30 performs the three primary tasks of this invention, 
preferably, but not necessarily, simultaneously: 

the absolute position and orientation of the second 
object, the cranium, in the fixed coordination system is 
detemined at least very frequently; 
3 5 the relationship between the absolute position and 

orientation of the second object with respect to the 
previously taken images of that object, particularly the 
inside structures of that object, is determined at least 
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very frequently; and 

the absolute position and orientation of the first 
object, the probe, is dsternined at least very frequently. 
-The accoiDplishment of these three tasks then permits the 
computer to acccmplish the three essential secondary tasks 
of this invention: 

to calculate the position and orientation of the 
first object, the probe, in relation to the second object, 
the cranium, even though the first object, or a portion of 
it, is out of the line of sight of either the surgeon or 
the sensors; 

to select the appropriate slice of the previously 
taken nodel of the interior of the second object which 
corresponds to the present tine position and orientation 
of the first object in relation to the present time 
position and orientation of the second object; and 

to display the appropriate slice of the previously 
taken image of the second object with the present time 
position and orientation of the first object correctly 
depicted thereon. 

Both the initial and the continual correlation 
determinations can be automatically initiated and updated 
by the computer 3 6 in some prsdeterrcined timed sequence or 
continuously. In fact, according to the most preferred 
aspect of this embodiment of this invention, the 
correlation phase is frequently, briefly from time to 
time, or even continuously, repeated, interspersed in 
between measurements in the operational phase or conducted 
simultaneously with the operational phase of the practice 
of this invention for the purpose of recalculating the 
linear transformations M and M' when the second object 
(such as a surgical patient) moves relative to the 
sensors. 

During normal operation, the tip coordinates are 
transformed in a step 44 using the transformation computed 
in step '45. The new transformed coordinates /relative to 
the image coordinate system, are used to determine the 
plane of some two-dimensional cross-section through the 
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three-dimensional image model 41 accessible in the 
accessible memory 43. The simplest method is simply to 
choose the existing diagnostic image plane located closest 
-to the probe tip's coordinates relative to the model 
coordinate system. 

In any case, a step 4 7 transforms the two- 
dimensional cross-sectional slice to a screen image and 
places a cursor on it to mark the locauion of the probe 
tip superimposed in the image. Scaling and viewing 
parameters determine how the image is displayed. Because 
the surgeon may not be able to simultaneously view the 
patient (object) and the computer display screen, the step 
47 should be controlled by the surgeon, such ' as for 
example by placing an activating button on the probe. 
15 Pressing the button can be the signal for freezing the 

image and the depicted position and orientation of the 
probe tip marker at that instant on the display screen. 

In a more complex embodiment of this invention, 
the computer system could generate and display on the 
20 screen a cut-away view at an arbitrary angle, for example, 

perpendicular to the direction the probe is pointing, 
using the data from multiple image slices. In simpler 
cases, the computer simply displays any one or more 
convenient image slices through the location of the probe 
25 tip. For example, the displayed slice might simply be the 
original CT slice which includes the location of the probe 
tip, or is closest to that location. In any case, the 
computer then causes the image a cursor at the current 
position of the probe tip to be displayed on this 
30 previously taken image of a slice through the second 
object. 

An alternative means, to record the location of 
the reference points in the coordinate space of the 
imaging apparatus during the imaging phase, employs an 
35 additional, separate instance of the three-dimensional 
position mensuration probe, sensors, control unit, and 
computer of the present invention. In order to implement 
this embodiment of this invention, the additional sensors 
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are permanently attached directly on the imaging 
apparatus. The additional probe measures the location of 
the reference points at the time of imaging, and the 
<idditianal control unit and computer determines and 
5 records their locations relative to the coordinate system 
of the imaging apparatus. The advantage of this approach 
is that the fiducial markers, that is the landmarks or 
reference pins, need not be within the limited cross- 
sectional slices visible to the imaging device, 

10 As an alternative to true three-dimensional 

images, standard x-ray radiographs from several distinct 
directions can be used to construct a crude model in lieu 
of the imaging phase described above. Radiographs from 
two or more directions are digitally scanned, and four 

15 non-coplanar reference points on them are identified with 

a cursor or light pen. In a correlation phase similar to 
that described above, these four points on the patient are 
digitized just prior to surgery. Then, during surgery, 
the location of the probe tip is projected onto the 

20 digitized computer images of the two-dimensional 

radiographs where the projection is uniquely defined by 
mapping and transferring tho reference point coordinates 
from the model coordinate system to the fixed sensor 
coordinate system. 

25 In a further embodiment of this invention, a 

videotape recording of the computer screen (as well as the 
direct view of the surgeon and patient) is used to help 
document the performance of the instant procedure. 
Radiation emitters may be present on more than one 

30 standard surgical tool such as the microscope, scalpel, 

forceps, and cauterizer, each of which thereby becomes, in 
effect, a probe. These emitters should be differentiated 
from each other in the same manner as aforesaid. 

The method and apparatus of the optical 

35 mensuration and correlation apparatus 10 of the present 
invention has'* been completely described. While some of 
the numerous modifications and equivalents of the system 
of this invention have been described herein, still other 
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modifications and changes will readily occur to those of 
ordinary skill in the art. For instance, the preferred 
embodiiDent described herein uses visible light, since 
"hunan operators can readily observe if the'light sources 
5 aire operative or whether they are causing troublesome 
reflections. Clearly, other wavelengths of 

electromagnetic radiation could be used without departing 
from the spirit and scope of the invention. Non-visible 
light, such as infrared or ultra-violet light, would have 

10 the advantage of not distracting the surgeon with flashing 
lights. Ultra-sound could be used conveniently. Other 
modifications to the detector "optics" and lenses are 
possible which would change, and possibly improve, the 
image characteristics on the detectors. For example, 

15 toroidal lenses could be used which are longitudinally 

curved along an arc with a radius equal to the focal 
length of the lens. Similarly, the surfaces of the 
photodetectors could also be curved, thus allowing the 
images of distant light .sources to remain in sharp focus, 

20 regardless of their positions. Numerous enhancements of 
the digital data are possible by suitably .programming. the 
computer. 

The most preferred aspects of this invention use 
electromagnetic radiation, and especially visible light, 

25 as the radiation from the emitters. This use of light for 
this function is a major improvement over the use in the 
prior art of audible sound emitters and detectors. 
However, prior art systems which are based on the use of 
sound emitters can be reprogrammed to carry out the 

30 operations to substantially continuously recorrelate the 
position and orientation of the second object during the 
surgical procedure, as they have been described herein. 
Thus, the movement of the second object can be at least 
frequently, if not continuously, tracked using sound 

35 emitters and detectors and suitable temperature 
compensation 'techniques. In this last regard, the 
aforementioned ability of the instant system to determine 
and transmit the temperature of the probe tip can be used 
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choice. The fact that the use of electromagnetic 
radiation, particularly light, esiitters and sensors is an 
improvement over the use of audible 'sound* emitters and 
5 sensors is not intended to be a limitation of the practice 
of continuously or frequently following the movement of 
the first or the second objects. That is an invention in 
and of itself using any emitter-sensor pair. 

It should be understood, however, that the 

10 transmission between emitters and sensors operate 
differently, and measure different things when 
electromagnetic radiation is used as compared to the use 
of sound, audible or ultrasonic. In the case of 
electromagnetic radiation, what is being measured is the 

15 angle that the radiation path makes between the emitter 

and the sensor relative to some arbitrary fixed line. By 
measuring all of these angles of the rays between the 
emitters and the sensors, conventional analytic geometry 
solutions will locate the points in space where the 

20 various emitters are. On the other hand, when sound 

radiation is used, what is being measured is the distance 
between each of the emitters and the sensors. Again, 
conventional analytic geometry solutions will precisely 
locate the point in space which is occupied by each 

25 emitters/sensor- While the casual observer, or the 
operator of the systems of this invention will not observe 
any difference in result, there is a marked difference in 
the way that result is achieved, and therefore this will 
necessitate a difference in the manner in which this 

30 system is programmed. 

The foregoing is illustrative of the principles 
of the invention. Since numerous modifications and 
changes will readily occur to those of ordinary skill in 
^he art, given the teachings of this specification, this 

35 invention is not limited to the exact construction and 

operation shown and described herein. Accordingly, all 
suitable modifications and equivalents that may be 
resorted to in light of disclosure of this specification 



wo 94/23 647 



PCT/US94/04293 



46 

are considered to fall within the scope of the invention 
as defined by the following claias. 
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VfHAT IS CLAIMED IS: 

1 " 1. A system for determining the present time 

2 location and orientation of a moveable first abject with 

3 respect to a moveable second object and for graphically 

4 indicating the corresponding position and orientation of 

5 said first object on a previously taken inage of said 

6 second object, which comprises: 

7 a present time three-dimensional fixed coordinate 

8 system; 

9 at least three radiation sensor means in fixed 

10 spacial relationship to said present time three- 

11 dimensional fixed coordinate system which is spaced from 

12 said movable objects; 

13 said moveable first and second objects located within 

14 said fixed coordinate system; 

15 a three-dimensional local coordinate system, which 

16 remains fixed with respoct to said sQcond object, but is 

17 movable, along with said second object, within said fixed 

18 coordinate system; 

19 at least three non-collinear radiation emitter means 

20 in fixed spacial relationship to said second object and at 

21 known coordinates in said local coordinate system; 

22 previously taken three-dimensional image data which 

23 geometrically describe said second object; 

24 at least two spaced apart radiation emitter means 

25 disposed on said first object; 

26 first movement means to move said first object 

27 relative to said second object and to said fixed 

28 coordinate system; 

29 second movement means to move said second object 
3 0 relative to said first object and to said fixed coordinate 
31 system; 

^2 means to transmit radiation between said radiation 

3 3 emitter means on said first and second objects and said 

34 radiation sensor means in fixed relation to said fixed 

35 coordination system; 
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35 means to distir.guish between radiation emitted from 

37 any one radiation emitter means from radiation einitred 

38 from all of the other radiation emitter means; 

39 - means to independently determine 'the location of each 

40 of said radiation emitter means on said first object as a 

41 function of said radiation being transferred between at 
4 2 least two of said radiation emitter means on said first 

43 object and said radiation sensor means in fixed relation 

44 to said fixed coordination system; 

4 5 means to determine the position and orientation of 

4 6 any point on said first object in relation to said fixed 

47 coordinate system by integrating the determined location 

48 of at least two of said radiation emitter means on said 

49 first object; 

50 means to independently determine the location of each 

51 of said radiation eniitter means associated with said 

52 second object as a function of said radiation being 

53 transferred between said radiation emitter means 

54 associated with said second object and said radiation 

55 sensor mean in fixed relation to said fixed coordination 

56 system; 

57 means to determine the position and orientation of 

58 said second object in said fixed coordinate system by 

59 integrating the determined location of said radiation 

60 emitter means associated with said second object; 

61 means to determine the position and orientation of 

62 said second object in said local coordinate system; 

63 means to orient said previously taken three 

64 dimensional image data of said second object in said local 

65 coordination system such as to match the present time 

66 position and orientation of said second object with the 

67 previously taken said image data; 

68 means to integrate the present time * determined 

69 position and orientation of said second object with the 

70 present time determined position and orientation of said 

71 first object In the same fixed coordinate system whereby 

72 integrating the present time determined position and 

73 orientation of said first object in relation to said 
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74 previously taken three dimensional inage data; 

75 means to, in present time, repeatedly determine the 

76 position and orientation of said first object with 

77 sufficient frequency to enable displaying the movement of 

78 said, first object in relation to said second object; and 

79 means to, in present tine, repeatedly determine the 

80 position and orientation of said second object with 

81 sufficient frequency to enable correlating a woved 

82 position and orientation of said second object with a view 

83 of said previously taken three dimensional inage data 

84 which is consistent with the moved position and 
8 5 orientation of said second object and allows for the 

86 correct present time positioning of an image corresponding 

87 to the present time position and orientation of said first 

88 object in relation to said previously taken image data; 

89 wherein, as a consequence of said repeated 

90 determinations and integrations of the positions and 

91 orientations of said first and second objects, and 

92 correlation of said position and orientation of said 

93 second object with said previously taken three dimensional 

94 image data, an iir.age representative of said first object 

95 is correctly positioned in present time on said previously 

96 taken image data regardless of the movement of said first 

97 and/or second objects in relation to said fixed coordinate 

98 system. 

1 2. A- system as claimed in claim 1 wherein said 

2 first and second objects intersect, which system further 

3 comprises means to display so much of said previously 

4 taken three dimensional image data as corresponds to the 

5 intersecting location of said first and second objects, 

6 and means to display said representation of said first 

7 object on said image display in its correct present time 
B position. 

1 ' 3. ' A system as claimed in claim 1 wherein said 

2 frequency of determining the position and orientation of 

3 said second object is about 0.01 to 100 tines the 
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4 frequency of determining the position and orientation of 

5 said first object. 

3^ . 4. A systeni as claimed in- clain 1 wherein the 

2 position and orientation of said second object, the 

3 integration of the position and orientation of said second 

4 object and said image data, and the • position and 

5 orientation of said first object are all determined 

6 substantially continuously. 

1 5. The system as claimed in claim 1 wherein 

2 said previously taken three dimensional image data 

3 comprise a succession of substantially two dimensional 

4 slices at previously determined, known locations and 

5 orientations through said second object. 

1 6. The system as claimed in claim 1 wherein 

2 said radiation emitter means are disposed on said first 

3 and second objects and said sensor means are spaced from 

4 said first and second objects and are disposed in fixed 

5 relationship to said fixed coordinate system. 

1 7. The system as claimed in claim 1 wherein 

2 said sensor means are disposed on said first and second 

3 objects and said emitter means are spaced from said first 

4 and second objects and are disposed in fixed relationship 

5 to said fixed coordinate system. 

1 8. The system as claimed in claim 1 wherein 

2 said radiation is electromagnetic radiation. 

1 9. The system as claimed in claim 6 wherein 

2 said radiation is light. 

1 10. The system as claimed in claim 9 wherein 

2 light of different wave lengths is emitted from each 

3 different emitter means. 
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1 11. The systen as claimed in claim 8 wherein 

2 said radiation is pulse radiated from different of said 

3 emitter means sequentially in a repeating order. 

1 12. The system as claimed in claim 1 wherein 

2 said radiation is sound. 

1 13. The systen as claimed in claim 1 wherein 

2 said radiation emitter means and said sensor means 

3 comprise means for detecting linear and rotational 

4 inertial motion. 

1 14, The system as claimed in claim 1 wherein 

2 said previously taken ixage data are taken by a means 

3 which is not suitable for use during a surgical procedure, 

1 15, The system as claimed in claim 1 wherein 

2 said first object is a surgical probe and said second 

3 object is a surgical patient, wherein said surgical probe 

4 is adapted to be partially inserted into said patient such 

5 that said inserted portion is not visible to an inserter 

6 thereof. 

1 16. The system as claimed in claim 15 wherein 

2 at least two emitter means are disposed on so much of said 

3 first object as is not inserted into said patient, and 

4 wherein said so disposed emitter means on said first 

5 object are a known distance from a tip of said probe which 

6 is adapted for insertion into said patient. 

1 17. The system as claimed in claim 1 wherein 

2 said image data are selected from the group consisting of 

3 magnetic resonant image data, computed tomography data, 

4 ultra sound data, or X-radiation data. 

1 18.'* The system as claimed in claim 1 wherein 

2 said second object is a patient, and said first object is 

3 an operating microscope, and wherein said image data are 
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4 projected onto the field of said operating microscope. 

1 19. The system as claimed in claim 1 wherein 

2 'said second object is a part of the anatomy of a" hunan. 

1 20. The systeni as claimed in claim 19 wherein 

2 the part of the anatomy is a head. 



1 21. The system as claimed in claim 19 wherein 

2 there are reference points on said second object, which 

3 are distinguishing anatomical features, which are in fixed 

4 spacial relationship to said emitter means associated with 

5 said second object. 

1 22. The system as claimed in claim 1 operated 

2 intermittently, but sufficiently frequently to depict the 

3 relative movement of said first object relative to said 

4 second object, and the movement of said second object 

5 relative to said fixed coordinate system. 

1 23. The system as claimed in claim 2 including 

2 means for monitoring in present time the movement of said 

3 first object in true relation to said previously taken 

4 three dimensional image data. 

1 24 . The system as claimed in claim 1 including 

2 means for selecting the slice of said previously taken 

3 image data which corresponds to the present time location 

4 of said first object such that a representation of said 

5 first object is correctly disposed on said image slice in 

6 present time. 

1 25. The system as claimed in claim 1 including 

2 means to freeze a display of a previously taken image and 

3 a representation of said first object for a finite period 

4 of time. 



1 



26. The system as claimed in claim 1 including 
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2 means to continuously display a series of slices of said 

3 previously taken three disiensional data which correspond 

4 to a changing position of said first object relative to 

5 -said second object, and which correctly reflect raovement 
5 of said second object relative to said fixed coordinate 
7 systen in present time. 

1 21. A system for determining the present tine 

2 location and orientation of a moveable first object with 

3 respect to a second object and for graphically indicating 

4 the corresponding position and orientation of said first 

5 object on a previously taken image of said second object, 

6 which comprises: 

7 a present time three-dimensional fixed coordinate 

8 systen; 

9 at least three radiation sensor means in fixed 

10 spacial relationship to said present time three- 

11 dimensional fixed coordinate system which are spaced from 

12 said objects; 

13 said first and second objects located within said 

14 fixed coordinate system; 

15 at least three non-collinear fiducial markers in 

16 fixed spacial relationship to said second object and at 

17 known coordinates in said local coordinate system; 

18 a three dimensional local coordinate system which is 

19 fixed in relation to said second object; 

20 previously taken three-dimensional image data which 

21 geometrically describe said second object including said 

22 fiducial markers; 

2 3 at least two spaced apart electromagnetic radiation 

24 emitter means disposed on said first object; 

25 first movement means to move said first object 

26 relative to said second object and to said fixed 

27 coordinate system; 

28 means to transmit radiation between said radiation 

29 emitter means' on said first object and said radiation 

30 sensor means in fixed relation to said fixed coordination 

31 system; 
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32 Tneans to distinguish radiation emitted from any one 

33 radiation emitter Tneans from radiation emitted from all of 

34 the other radiation emitter means; 

35 - ' means to independently determine" the location of each 

36 of said radiation emitter sieans on said first object as a 

37 function of said radiation being transferred between at 

38 least two of said radiation emitter means on said first 

3 9 object and said radiation sensor means in fixed relation 

40 to said fixed coordination system? 

41 means to determine the position and orientation of 

4 2 any point on said first object in relation to said fixed 
4 3 coordinate system by integrating the determined location 
4 4 of at least two of said radiation emitter means on said 
4 5 first object; 

4 6 means to independently determine the location of each 

4 7 of said fiducial markers on said second object in said 
4 8 • fixed coordinate system; 

49 means to determine the position and orientation of 

50 said second object in said fixed coordinate system by 

51 integrating the determined location of said fiducial 

52 markers on said second object; 

53 means to orient said previously taken three 

54 dimensional image data of said second object such as to 

55 match the present time position and orientation of said 

56 second object with the previously taken said image data; 

57 means to integrate the present time determined 

58 position and orientation of said second object with the 

59 present time determined position and orientation of said 

60 first object in the same fixed coordinate system whereby 

61 integrating the present time determined position and 

62 orientation of said first object in relation to said 

63 previously taken three dimensional image data; and 

64 means to, in present time, repeatedly determine the 

65 positions and orientations of said first object with 

66 sufficient frequency to enable displaying the movement of 

67 said first object in relation to said second' object ; 

68 wherein, as a consequence of said repeated 

69 determinations and integrations of the positions and 
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70 orientations of said first object, and correlation of said 

71 positions and orientations of said second object with said 

72 previously taken three dimensional iinage data, an image 

73 -representative of said first object * is correctly 

74 positioned in present time on said previously taken image 

75 data regardless of the movement of said first object in 

76 relation to said fixed coordinate system. 

1 28. A system as claimed in claim 27 wherein 

2 said first and second objects intersect, which system 

3 further comprises means to display so much of said 

4 previously taken three dimensional image data as 

5 corresponds to the intersecting location of said first and 

6 second objects, and means to display said representation 

7 of said first object on said image display in its correct 

8 present time position. 

1 29. A systeiu as claimed in claim 27 wherein 

2 the integration of the position and orientation of said 

3 second object and said image data, the position and 

4 orientation of said first object, and the correlation of 

5 the position and orientation of said first object with 

6 said image data are all determined substantially 

7 continuously. 

1 30. The system as claimed in claim 27 wherein 

2 said previously taken three dimensional image data 

3 comprise a succession of substantially two dimensional 

4 slices through said second object. 

1 31. The system as claimed in claim 27 wherein 

2 said radiation emitter means are disposed on said first 

3 object and said sensor means are spaced from said first 

4 and second objects and are disposed in fixed relationship 

5 to said fixed coordinate system. 

1 32. The system as claimed in claim 27 wherein 

2 said sensor means are disposed on said first object and 
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3 said radiation emitter neans are spaced from said first 

4 object and are disposed in fixed relationship to said 

5 fixed coordinate systen. 

1 33. The systen as claiuied in claim 27 wherein 

2 said radiation is light. 

1 34, The systen as claiiaed in claim 33 wherein 

2 said radiation is visible light. 

1 35. The system as claimed in claim 33 wherein 

2 light of different wave lengths is emitted from each 

3 different emitter means. 

1 36. The system as claimed in claim 33 wherein 

2 said light is pulse radiated from different of said 

3 radiation emitter means sequentially in a repeating order. 

1 37. The system as claimed in claim 27 wherein 

2 said radiation emitter means and said sensor means 

3 comprise means for detecting linear and rotational 

4 inertial notion. 

1 38. The system as claimed in claim 27 wherein 

2 said previously taken image data are taken by a means 

3 which is not suitable for use during a surgical procedure. 

1 39- The system as claimed in claim 27 wherein 

2 said first object is a surgical probe and said second 

3 object is a surgical patient, wherein said surgical probe 

4 is adapted to be partially inserted into said patient such 

5 that said inserted portion is not visible to an inserter 

6 thereof. 

1 40. The system as claimed in claim 39 wherein 

2 at least two emitter means are disposed on so much of said 

3 first object as is not inserted into said patient, and 

4 wherein said so disposed emitter means on said first 
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5 object are a kno%7i distance froT; a tip of said probe which 

6 is adapted for insertion into said patient. 

3_ - 41, The system as claimed *in claim 27 wherein 

2 said image data are selected from the group consisting of 

3 magnetic resonant image data, computed tonography data, 

4 ultra sound data, or X-radiation data. 

1 <2. The system as claimed in claim 27 wherein 

2 said second object is a patient, and said first object is 

3 an operating microscope, and wherein said image data are 

4 projected onto the field of said operating microscope. 

1 43. The system as claimed in claim 27 wherein 

2 said second object is a part of the anatomy of a human. 

1 44. The system as claimed in claim 43 wherein 

2 the part of the anatomy is a head. 

1 45. The system as claimed in claim 43 wherein 

2 there are reference points on said second object, which 

3 are distinguishing anatomical features, which are in fixed 

4 spacial relationship to said second object. 

1 46. The system as claimed in claim 27 operated 

2 intermittently, but sufficiently frequently to depict the 

3 relative movement of said first object relative to said 

4 second object. 

1 47. The system as claimed in claim 28 including 

2 means for monitoring in present time the movement of said 

3 first object in true relation to said previously taken 

4 three dimensional image data. 

1 48- The system as claimed in claim 27 including 

2 means for selecting the slice of said previously taken 

3 image data which corresponds to the present time location 

4 of said first object such that a representation of said 
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5 first object is correctly disposed on said inage slice in 

6 present tice. 

1 - 49. The system as clained in claim 27 including 

2 neans to freeze a display of a previously taken inage and 

3 a representation of said first object for a finite period 

4 of tine. 

1 50. The system as claimed in claim 27 including 

2 means to continuously display a series of slices of said 

3 previously taken three dimensional data which correspond 

4 to a changing position of said first object relative to 

5 said second object, and which correctly reflect the 

6 position and orientation of said second object relative to 

7 said fixed coordinate system in present time. 

1 51. In a system for determining the present 

2 time location and orientation of a moveable first object 

3 with respect to a second object and for graphically 

4 indicating the corresponding position and orientation of 

5 said first object on a previously taken image of said 

6 second object, which comprises: 

7 a present time three-dimensional fixed coordinate 

8 system; 

9 at least three radiation sensor means, in fixed 

10 spacial relationship to said present time three- 

11 dimensional fixed coordinate system, which are spaced from 

12 said objects; 

13 said first and second objects located within said 

14 fixed coordinate system; 

15 a three dimensional local coordinate system which is 

16 fixed in relation to said second object; 

17 at least three non-collinear fiducial' markers in 

18 fixed spacial relationship to said second object and at 

19 known coordinates in said local coordinate system; 

20 previously taken three-dimensional image data which 

21 geometrically describe said second object including said 

22 fiducial markers in said fixed coordinate system; 
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23 at least two spaced apart radiation emitter means 

24 disposed on said first object; 

25 first nover^ent neans to move said first object 

26 'relative to said second object; 

27 TTieans to transmit radiation between said radiation 

28 emitter means on said first object and said radiation 

29 sensor means in fixed relation to said fixed coordinate 

30 system; 

31 moans to distinguish radiation emitted froa any one 

32 radiation emitter means from radiation emitted from all of 

33 the other radiation emitter means; 

34 means to independently determine the location of each 

35 of said radiation emitter means on said first object as a 

3 6 function of said radiation being transferred between at 

37 least two of said radiation emitter means on said first 

38 object and said radiation sensor means in fixed relation 

39 to said fixed coordinate system; 

4 0 means to determine the position and orientation of 

41 any point on said first object in relation to said fixed 

42 coordinate system by integrating the determined location 
4 3 of at least two of said radiation emitter means on said 
44 first object; 

4 5 means to independently determine the location of each 

46 of said fiducial markers on said second object; 

4 7 means to determine the position and orientation of 

48 said second object in said fixed coordinate system by 

4 9 integrating the- determined location of said fiducial 

50 markers on said second object into said fixed coordinate 

51 system; 

52 means to orient said previously taken three 

53 dimensional image data of said second object such as to 

54 match the present time position and orientation of said 

55 second object with the previously taken said image data; 

56 means to integrate the present time determined 

57 position and orientation of said second object with the 

58 present "time "determined position and orientation of said 

59 first object in the same fixed coordinate system whereby 
6.0 integrating the present time determined position and 
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61 orientation of said first object in relation to said 

62 previously taken three dimensional ir^age data; and 

63 neans to, in present time, repeatedly determine the 

64 - position and orientation of said*- first object with 

65 sufficient frequency to enable displaying the movement of 

66 said first object in relation to said second object; 

67 wherein, as a consequence of said repeated 

68 determinations and integrations of the positions and 

69 orientations of said first object, and correlation of said 

70 position and orientation of said second object with said 

71 previously taken three dimensional image data, an image 

72 representative of said first object is correctly 

73 positioned in present time on said previously taken image 

74 data regardless of the movement of said first object in 

75 relation to said fixed coordinate system; 

76 the improvement which comprises said radiarion 

77 being electromagnetic radiation. 

1 52. In a system for determining the present 

2 time location and orientation of a moveable first object 

3 with respect to a moveable second .object and for 

4 graphically indicating the corresponding position and 

5 orientation of said first object on a previously taken 

6 image of said second object, which comprises: 

7 a presenr time three-dimensional fixed coordinate 

8 system; 

9 at least three radiation sensor means in fixed 

10 spacial relationship to said present time three- 

11 dimensional fixed coordinate system which is spaced from 

12 said movable objects; 

13 said moveable first and second objects located within 

14 said fixed coordinate system; 

15 a three-dimensional local coordinate system, which 

16 remains fixed with respect to said second object, but is 

17 movable, along with said second object, within said fixed 

18 coordinate system; 

19 at least three non-collinear fiducial marker means in 

20 fixed spacial relationship to said second object and at 
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21 known coordinates in said local coordinate systen; 

22 previously taken three-dinensional image data which 

23 geometrically describe said second object; 

24 - at least two spaced apart radiation eniitter means 

25 disposed on said, first object; 

26 first movement means to move said first object 

27 relative to said second object and to said fixed 

28 coordinate system; 

29 second movement means to move said second object 
3 0 relative to said first object and to said fixed coordinate 

31 system; 

32 means to transmit radiation between said radiation 

33 emitter means on said first object and said radiation 

34 sensor means in fixed relation to said fixed coordination 

35 system; 

3 6 means to distinguish radiation emitted from any one 

37 radiation emitter means from radiation emitted from all of 

38 the other radiation emitter means; 

39 means to independently determine the location of each 

40 of said radiation emitter means on said first object as a 

41 function of said radiation being transferred between at 

42 least two of said radiation emitter means on said first 

4 3 object and said radiation sensor means in fixed relation 

44 to said fixed coordination system; 

45 means to determine the position and orientation of 
4 6 any point on said first object in relation to said fixed 
47 coordinate system by integrating the determined location 
4 8 of at least two of said radiation emitter means on said 
4 9 first object; and 

50 in present time, repeatedly determining the position 

51 and orientation of said first object with sufficient 

52 frequency to enable displaying the movement of said first 

53 object in relation to said second object; 

54 the improvement which comprises: 

55 radiation emitter means associated in fixed, known 

56 spacial relationship to said fiducial markers; 

57 means to determine the location of said fiducial 

58 markers on said second object as a function of said 
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59 radiation being transferred between said radiation emitter 

60 means associated with said second object and said 

61 radiation sensor means in fixed relation to said fixed 

62 - coordinate systeia; 

63 means to determine the position and orientation of 

64 said second object in said fixed coordinate system by 

65 integrating the determined location of said radiation 

66 emitter means associated with said second object; 

67 means to determine the position and orientation of 

68 said second object in said local coordinate system; 

69 means to orient said previously taken three 

70 dimensional image data of said second object in said local 

71 coordination system such as to match the present time 

72 position and orientation of said second object with the 

73 previously taken said image data; 

74 means to integrate the present time determined 

75 position and orientation of said second object with the 

76 present time determined position and orientation of said 

77 first object in the same fixed coordinate system whereby 

78 integrating the present time determined position and 

79 orientation of said first object in relation to said 

80 previously taken three dimensional image data; and 

81 means to, in present time, repeatedly determine the 

82 position and orientation of said second . object with 

83 sufficient frequency to enable correlating a moved 

84 position and orientation of said second object with a view 

85 of said previously taken three dimensional image data 

86 which is consistent with the moved position and 

87 orientation of said second object and allows for the 

88 correct present time positioning of an image corresponding 

89 to the present time position and orientation of said first 

90 object in relation to said previously taken image data. 

1 53, A method of determining the present time 

2 location and orientation of a moveable first object with 

3 respect to a moveable second object and for graphically 

4 indicating the corresponding position and orientation of 

5 said first object on a previously taken image of said 
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6 second object, which comprises: 

7 defining a present tine three-dimensional global 

8 fixed coordinate system; 

9 - * providing at least three radia1:ion sensor means in 

10 fixed spacial relationship to said present time three- 

11 dimensional fixed coordinate system which is spaced from 

12 said movable objects? 

13 disposing said moveable first and second objects 

14 located within said fixed coordinate system; 

15 defining a three-dimensional local coordinate system, 

16 which remains fixed with respect to said second object, 

17 but is movable, along with said second object, within said 

18 fixed coordinate system; 

19 providing at least three non-coil inear radiation 

20 eniitter means in fixed spacial relationship to said second 

21 object and at known coordinates in said local coordinate 

22 system; 

23 taking, at a previous time, three-dimensional image 

24 data which geometrically describe said second object; 

25 providing at least two spaced apart radiation emitter 

2 6 means disposed on said first object; 

27 providing first movement means to move said first 

28 object relative to said second object and to said fixed 

29 coordinate system; 

30 providing second movement means to move said second 

31 object relative to said first object and to said fixed 

3 2 coordinate system; 

33 transmitting radiation between said radiation emitter 

34 means on said first and second objects and said radiation 

35 sensor means in fixed relation to said fixed coordination 
3 6 system; 

37 distinguishing radiation emitted from any one 

38 radiation emitter means from radiation emitted from all of 

39 the other radiation emitter means; 

40 independently determining the location of each of 

41 said radiation emitter means on said first' object as a 

42 function of said radiation being transferred between at 

43 least two of said radiation emitter means on said first 
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44 object and said radiation sensor neans in fixed relation 
4 5 to said fixed coordination system; 

45 detemining the position and orientation of any point 
47 - on said first object in relation to 'said fixed coordinate 

46 system by integrating the determined location of at least 

49 two of said radiation emitter means on said first object; 

50 independently determining the location of each of 

51 said radiation emitter means associated with said second 

52 object as a function of said radiation being transferred 

53 between said radiation emitter means associated with said 

54 second object and said radiation sensor means in fixed 

55 relation to said fixed coordination system; 

56 determining the position and orientation of said 

57 second object in said fixed coordinate system by 

58 integrating the determined location of said radiation 

59 emitter means associated with said second object; 

50 determining the position and orientation of said 

61 second object in said local coordinate system; 

62 orienting said previously taken three dimensional 

63 image data of said second object in said local 

64 coordination system such as to natch the present time 

65 position and orientation of said second object with the 

66 previously taken said image data; 

67 integrating the present time derermined position and 

68 orientation of said second object with the present time 

69 determined position and orientation of said first object 

70 in the same fixed coordinate system whereby integrating 

71 the present time determined position and orientation of 

72 said first object in relation to said previously taken 

73 three dimensional image data; 

74 in present time, repeatedly determining the position 

75 and orientation of said first object with sufficient 

76 frequency to enable displaying the movement of said first 

77 object in relation to said second object; and 

78 in present time, repeatedly determining the position 

79 and orientation of said second object with sufficient 

80 frequency to enable correlating a moved position and 

81 orientation of said second object with a view of said 
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82 previously taken three dinensional image data which is 

8 3 consistent with the moved position and orientation of said 

84 second object and allows for the correct present time 

85 - positioning of an image corresponding to the present time 

86 position and orientation of said first object in relation 

87 to said previously taken image data; 

88 wherein, as a consequence of said repeatedly 

89 determining and integrating the positions and orientations 

90 of said first and second objects, and correlating said 

91 position and orientation of said second object with said 

92 previously taken three dimensional image data, positioning 

93 an image representative first object correctly in present 

94 tine on said previously taken image data regardless of the 

95 movement of said first and/or second objects in relation 

96 to said fixed coordinate system. 

1 54. A method as claimed in claim 53 wherein 

2 said first and second objects intersect, which method 

3 further comprises displaying so much of said previously 

4 taken three dimensional image data as corresponds to the 

5 intersecting location of said first and second objects, 

6 and displaying said representation of said first object on 

7 said image display in its correct present time position. 

1 55. A method as claimed in claim 53 wherein 

2 including determining the position and orientation of said 

3 second object at a frequency which is about 0.01 to 100 

4 times the frequency of determining the position and 

5 orientation of said first object. 

1 56. A method as claimed in claim 53 including 

2 substantially continuously determining the position and 

3 orientation of said second object, the integration of the 

4 position and orientation of said second object and said 

5 image data, and the position and orientation of said first 

6 object.- 



1 



57. The method as claimed in claim 53 including 
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2 taking three dimensional image data which comprise a 

3. succession of substantially two diaiensional slices at a 

4 previous tine at known locations and orientations through 

5 - said second object. 

1 58. The method as claimed in claim 53 including 

2 disposing said radiation emitter means on said first and 

3 second objects, spacing said sensor means a distance from 

4 said first and second objects, and disposing said sensor 

5 means in fixed relationship to said fixed coordinate 

6 system. 

1 59- The method as claimed in claim 53 including 

2 disposing said sensor means on said first and second 

3 objects, spacing said eititter means a distance from said 

4 first and second objects, and disposing said radiation 

5 emitters in fixed relationship to said fixed coordinate 

6 system. 

1 60, The method as claimed in claim 53 wherein 

2 said radiation is electromagnetic radiation 

1 61. The method as claimed in claim 60 wherein 

2 said radiation is light. 

1 62. The method as claimed in claim 61 including 

2 emitting light of different wave lengths from each 

3 different emitter means. 

1 63. The method as claimed in claim 60 including 

2 pulse radiating said radiation from different of said 

3 emitter means sequentially in a repeating order. 

1 64. The method as claimed in claim 53 wherein 

2 said radiation is sound. 

1 65. The method as claimed in claim 53 including 

2 detecting linear and rotational inertial motion through 
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3 said radiation enitter nieans and said sensor neans, 

1 66. The method as claimed in claim 53 including 

2 " taking said previously taken iinage data by a neans which 

3 is not suitable for use during a surgical procedure. 

1 67. The x:\ethcd as claimed in claim 53 wherein 

2 said first object is a surgical probe and said second 

3 object is a surgical patient, and partially inserting said 

4 surgical probe within said patient such that said inserted 

5 portion is not visible to an inserter thereof. 

1 68. The method as claimed in claim 67 including 

2 disposing at least two emitter means on so much of said 

3 first object as is not inserted into said patient, and 

4 disposing said exposed portion of said emitter means on 

5 said first object a known distance from a tip of said 

6 probe which is adapted for insertion into said patient, 

1 69. The method as claimed in claim 53 including 

2 selecting said image data from the group consisting of 

3 magnetic resonant image data, computed tomography data, 

4 ultra sound data, or X-radiation data. 

1 70. The method as claimed in claim 53 wherein 

2 said second object is a patient, and said first object is 

3 an operating microscope, and including projecting said 

4 image data onto the field of an operating microscope. 

1 71. The method as claimed in claim 53 wherein 

2 said second object is a part of the anatomy of a human. 

1 72. The method as claimed in claim 71 wherein 

2 the part of the anatomy is a head. 



1 
2 
3 



73'. The system as claimed in claim 71 wherein 
there are fiducial reference points on said second object, 
which are distinguishing anatomical features, which are in 
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4 fixed spacial relationship to said emitter means 

5 associated with said second object. 

1 - 74. The method as clained'-in claiai 53 which is 

2 operated intermittently, but sufficiently frequently to 

3 depict the relative movement of said first object relative 

4 to said second object, and the movement of said second 

5 object relative to said fixed coordinate system. 

1 75. The method as claimed in claim 54 including 

2 monitoring, in present tine, the movement of said first 

3 object in true relation to said previously taken three 

4 dimensional image data. 

1 76. The method as claimed in claim 53 including 

2 selecting the slice of said previously taken image data 

3 which corresponds to the present tine location of said 

4 first object, and correctly positioning a representation 

5 of said first object on said selected image slice in 

6 present time. 

1 77. The method as claimed in claim 53 including 

2 freezing a display of a previously taken image and a 

3 representation of said first object thereon for a finite 

4 period of time. 

1 7B. The method as claimed in claim 53 including 

2 continuously displaying a series of slices of said 

3 previously taken three dimensional data which correspond 

4 to a changing position of said first object relative to 

5 said second object, and correctly reflecting thereon the 

6 movement of said second object relative to said fixed 

7 coordinate system in present time. 
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